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CHAPTER 1: Introduction 


Background 


The Canadian Labour Force Survey (LFS) was introduced 
following the Second World War to satisfy a need for 
reliable and timely data on the labour market. Information 
was urgently required on the massive labour market 
changes during the transition from a war-time to a peace- 
time economy. The survey was designed to provide 
estimates of employment and unemployment at the 
regional and national level. 


The LFS began as a quarterly survey in 1945 and became 
a monthly survey in 1952. In 1960, the Interdepartmental 
Committee on Unemployment Statistics recommended 
that the LFS be designated the source of the official 
measure of unemployment in Canada. This endorsement 
was followed by a demand for a broader range of labour 
market statistics, including more detailed regional data. 
The information generated by the survey has expanded 
considerably over the years and now provides a detailed 
picture of the Canadian labour market. 


Concepts and Outputs 


The LFS is the only official source of monthly estimates 
of total employment (both paid workers and the self- 
employed, as well as full-time and part-time workers) and 
unemployment. Key rates published monthly include the 
unemployment rate (defined as the number of unemployed 
people divided by the size of the labour force), the 
employment rate (defined as the number of employed 
people divided by the total population, i.e., the number of 
people aged 15 years and older), and the participation rate 
(the proportion of the population that is either employed 
or unemployed). It is a major source of information on the 
personal characteristics of the working-age population, 
including age, marital status, educational attainment and 
family characteristics. 


Employment estimates include detailed breakdowns by 
industry and occupation, job tenure, and usual and actual 
hours worked. The survey incorporates questions 
permitting analyses of many topical issues, such as 
involuntary part-time employment, multiple job-holding, 
and absence from work. 


Unemployment estimates are produced by industry and 
occupation as well as by duration of unemployment, type 


of work sought, and activity before looking for work. 
Information is also available on the recent labour market 
involvement of persons not in the labour force. For a full 
description of the content of the LFS questionnaire, see 
Statistics Canada (1998)'. 


As part of the redesign of the LFS, a new questionnaire 
was introduced effective January 1997. The new 
questionnaire adds new content, including wages and 
union membership. Sunter et al. (1995) give a detailed 
description of the process of redesigning the LFS 
questionnaire. 


In addition to providing national and provincial estimates, 
the Labour Force Survey also releases estimates for 
subprovincial areas such as Employment Insurance 
Economic Regions (EIERs) and Census Metropolitan 
Areas (CMAs). In recent years, estimates of the standard 
labour market indicators have also been tabulated for 
small areas, such as Census Divisions (CDs) and Canada 
Employment Centres, using special estimation techniques. 
LFS estimates are used by both federal and provincial 
governments in allocating funds and other resources 
among various political and administrative jurisdictions. 


Standard estimates from the LFS are published monthly in 
Labour Force Information (Catalogue 71-001-PPB). A 
variety of labour market information is also available 
from CANSIM, Statistics Canada’s electronic database 
and retrieval system. Over nine thousand time series on 
this database are updated monthly by the LFS. 


Beginning in 1997, the quarterly Labour Force Update 
(Catalogue 71-005-XPB) provides an in-depth look at a 
variety of subjects central to analysis of the labour market. 
Each issue has a specific focus, and topics such as hours 
of work, youths in the labour market and wages will be 
assessed on a regular basis. 


The LFS also produces an annual Labour Force 
Historical Review on CD-ROM (Catalogue 
71F0004XCB) containing comprehensive data in both 
cross-sectional and time series format from 1976 to the 
current year. 


: References are given before Appendix A. 


A great deal more information is available from the survey 
than can be published regularly. Requests for custom 
tabulations are filled on a cost-recovery basis. 


An Overview of the Survey 


Target population. The LFS covers 98 percent orate 
Canadian population. The survey excludes from its 
coverage the Northwest Territories and residents of Indian 
reserves and Crown lands. Also excluded are inmates of 
institutions and full-time members of the Canadian Armed 
Forces since both groups are considered to be outside the 
labour market. The survey establishes the labour force 
status of all members of selected households who are 15 
years of age and older. 


Sample size. At the time of this writing, the target sample 
size for the LFS is 52,350 households. However, it has 
varied over time. During the 1970s redesign, the size of 
the sample was increased from 35,000 households per 
month to 55,000 households to meet the increasing 
demand for more reliable and detailed data at the 
provincial level. The LFS has also occasionally 
experienced reductions in its monthly samples. Following 
two such reductions in the 1980s, the sample comprised 
about 47,000 households. In 1989, the sample was 
increased by 16,500 households to 63,000 households. 
The aim of the increase was to produce better estimates 
for Unemployment Insurance Regions. The sample was 
decreased to about 59,000 households in 1993. Based on 
the efficiency gains from the new design, the sample was 
decreased further to 52,350 households, effective July 
1995. With legislative changes introduced in June 1996, 
the boundaries of the UI regions were revised and the 
regions were renamed Employment Insurance Economic 
Regions (EIERs). 


Sample rotation. The LFS follows a rotating panel sample 
design, in which households remain in the sample for six 
consecutive months. The sample is split into six 
representative sub-samples, and each month one-sixth of 
the sample is replaced after it has completed its stay in the 
survey. This results in a five-sixths month-to-month 
sample overlap, which makes the design efficient for 
estimating month-to-month changes. The rotation after six 
months prevents undue respondent burden for households 
that are selected for the survey. 


Data collection. The LFS reference week is normally the 
week containing the fifteenth day of the month. Data 
collection for the LFS is carried out during the week 
following reference week. Statistics Canada employs 


about 850 interviewers, including 80 senior interviewers, 
across the country, and their data collection activities are 
managed from five Regional Offices (ROs). LFS 
interviews are conducted in person in the first month and 
by telephone in the five subsequent months. The 
questionnaires are completed by the interviewer using 
Computer Assisted Interviewing (CAI) on portable 
computers. The collected data are uploaded daily by the 
interviewer to the RO during the interview week and then 
transmitted to Ottawa for processing. The data collection, 
data processing and dissemination processes are 
streamlined and efficiently run. As a result, Statistics 
Canada publishes monthly LFS estimates just 13 days 
after the end of the interview week. 


Due to the importance of the statistics produced from the 
survey and the complexities of the operations involved, 
various quality evaluation and control programs designed 
for different phases of the survey are monitored regularly. 


Survey redesign. Following each decennial population 
census, the LFS has undergone a sample redesign to 
reflect changes in population characteristics and to 
respond to changes in the information needs to be 
satisfied by the survey. The redesign that took place 
following the 1971 census was the biggest redesign prior 
to the current one. Not only was the sample design 
modified, but also, substantial changes were made to the 
questionnaire, and a new data processing infrastructure 
was introduced. The redesign that took place following 
the 1981 census focused mainly on updating the sample 
design itself. The current redesign is again a major one, 
encompassing all aspects of the survey: computer assisted 
interviewing has been introduced, the sample design has 
changed, data processing and dissemination systems have 
undergone a complete overhaul, and the questionnaire has 
been revised substantially. 


Objectives of the Sample Redesign 


The current sample redesign program culminated with the 
introduction of a new sample at the end of 1994. The 
program included extensive consultation involving the 
reassessment not only of the survey’s principal role as a 
provider of current labour market information, but also of 
its use as a central vehicle for conducting household 
surveys within Statistics Canada. 


A redesign provides an opportunity to update the 
sampling frame, its stratification and the allocation of the 
sample to reflect changes in population size and 
distribution. The design in place until the end of 1994 


used 1981 census geography and the corresponding 1981 
census counts to select the sample and derive estimation 
weights. Since many standard geographical units change 
with every census, each redesign enables the LFS to adopt 
the most recent definitions of these units. 


A goal in previous redesigns had been to make the LFS 
frame, sample and systems more flexible for other surveys 
since many household surveys conducted by Statistics 
Canada use these to meet their own needs. This goal was 
equally important in the current redesign. A related goal 
common to the last two redesigns was to take advantage 
of changes in technology and field operations to simplify 
the design. 


A Note on UI Regions and EIE Regions. It was noted 
above in the overview that in 1995, Unemployment 
Insurance (UI) Regions were replaced by Employment 
Insurance Economic Regions, or EIERs. For convenience, 
we will use the new name throughout this document even 
though the sample redesign was based on the UI Regions. 
The supplementary sample of 16,500 households that was 
introduced in 1989 to improve estimates for EIERs will be 
referred to as the EJ sample. The remaining sample, which 
currently consists of 35,850 households, will be referred 
to as the core sample. 


A new goal for this redesign was to use the core sample to 
satisfy the requirements for national and provincial 
estimates while using the EI sample to optimize estimates 
for the EJE regions. How this was achieved will be 
explained in chapter 2. 


Scope of the Report 


This report is a reference on the methodological aspects 
of the Labour Force Survey. The survey design, 
estimation methodology and data quality are discussed in 
detail, with supplementary references given where 
appropriate. This document is complemented by a 
separate report entitled Guide to the Labour Force Survey 
(available on the internet at 
www.statcan.ca/english/concepts/labour/index.htm) which 
emphasizes subject matter issues and the data that is 
available from the survey. 


There are five additional chapters in this report. Chapter 
2 describes stratification and the formation of sampling 
units such as clusters. Chapter 3 gives a detailed 
discussion of sample allocation, sample selection in urban 
and rural areas and rotation of the sample over time. The 


use of the LFS sample and frame by other household 
surveys is discussed briefly in Chapter 4. 


Chapter 5 presents a detailed description of the Labour 
Force Survey’s weighting and estimation system, 
including the handling of nonresponse. Finally, the LFS 
has an extensive data quality monitoring program which 
is described in Chapter 6. 


Note: A list of abbreviations used in this document is 
given in Appendix B. A diagram summarizing the new 
Labour Force Survey design is given in Appendix C. 


CHAPTER 2: Stratification and Sampling Unit Formation 


Canada’s population lives in various geographical areas 
such as provinces and regions which have standard 
definitions. Survey samplers usually partition the 
population further into strata, from which samples are then 
selected independently. If the population units in each 
stratum are relatively homogeneous, then the size of the 
sample required to obtain estimates of a specified 
precision will be much smaller than the size needed under 
an unstratified design. Stratification has other useful 
properties: different designs, sampling schemes and 
estimation methods can be used in different strata, 
selective updating of the design only in strata that have 
undergone rapid change can be done, and strata can 
sometimes be useful operational units. In this section, we 
describe the stratification used by the Labour Force 
Survey. 


Most surveys use two types of strata: (i) existing standard 
geographical units such as metropolitan areas and (11) 
strata formed by combining smaller units such as census 
enumeration areas according to an objective criterion. 
The standard geographical units used as strata by the LFS 
are described first. 


With the exception of Prince Edward Island, each 
province is divided into Economic Regions (ERs). The 
LFS has used ERs as primary strata since the 1960s. In 
the early stages of the current redesign, the definitions of 
the ERs were reviewed in consultation with the provinces. 
There are now 72 ERs in Canada. 


In previous LFS designs, ERs were the only subprovincial 
regions taken into account when designing the survey. In 
1989, Human Resources Development Canada (HRDC) 
started funding an increase in the LFS sample of 16,500 
households per month. This sample is used to improve 
the labour force estimates produced for the former 61 
(now 53) Employment Insurance Economic Regions 
(EIERs). Thus, in the new design, both ERs and EIERs 
were taken into account in stratification. In allocation, 
relatively less attention was paid to improving estimates 
for ERs since the core sample was allocated primarily 
with provincial and national estimates in mind, while the 
extra sample funded by HRDC was directly targeted to 
EIE regions. 


The two sets of regions, while roughly equal in number, 
were defined for different purposes and do not usually 


coincide. To deal with both sets of regions in the new 
LFS design, the intersections of the regions were 
considered as basic strata. Due to a degree of overlap 
between the EIJERs and ERs, there were 133 such 
intersections. 


A third set of regions, the Census Metropolitan Areas, has 
also been respected by the LFS stratification in the current 
and previous designs. CMAs are urban areas whose 
population was at least 100,000 in the most recent census. 
All CMAs are also EJERs. 


In previous designs, by the time a new design was in 
place, the CMA definitions it used were already four years 
old. For example, the previous design was fully in place 
in March 1985 and used CMA definitions from the June 
1981 census. In the new design, the work to define CMAs 
for the 1996 Census was started early to accommodate the 
LFS redesign which is based on these new CMA 
definitions. The official 1996 CMAs differ from the ones 
used initially by the LFS since legal changes in municipal 
boundaries must be taken into account. The differences 
are minor, and adjustments are made to the LFS to respect 
the final CMA definitions. The 1996 CMAs have also 
been adopted as EJERs. 


Within the larger geographical strata, more detailed strata 
were formed without regard to geographical constraints. 
This was done using the same method used in the 
previous design, namely a clustering algorithm due to 
Friedman and Rubin (1967) and modified for the LFS by 
Drew et al. (1985). The algorithm partitions units into 
strata that are as homogeneous as possible for several 
variables by minimizing a within-group weighted sum of 
squares. The sums of squares are computed to reflect the 
sampling of units with probability proportional to size. 
Different weights can be assigned to different variables if 
desired. More details of the algorithm can be found in 
Drew et al. (1985) and in Singh et al. (1990). 
Stratification using this algorithm will be referred to as 
optimal stratification. 


Stratification variables. The variables used in the 
stratification program, given below, include all those used 
in the previous LFS design. However, the new design 
uses a more detailed breakdown of employment by 
industry, specifically for the manufacturing and services 
sectors. The only completely new stratification variables 


are based on mother tongue. For each stratification unit, 
three language variables were coded: the number of 
people who gave English, French or Other (i.e., any other 
language) as their mother tongue. Finally, the income 
variable was given three times the weight of the other 
stratification variables. 


Data from the 1991 census were used for stratification. 
The following variables were used. 


Employed in 
agriculture 
forestry and fishing 
mining 
manufacturing - consumables 
manufacturing - rubber, plastics, leather 
manufacturing - textiles and clothing 
manufacturing - furniture, pulp and paper, 

printing, wood 

manufacturing - metals and minerals 
manufacturing - petrochemical, chemical 
construction 
transportation 
services - trade 
services - financial 
services - personal/business 
services - government 

total employed 


total income 

population aged 15+ 

population aged 15-24 

population aged 55+ 

number of one-person households 
number of two-person households 
number of owned dwellings 

total gross rent 

population with high school education 
mother tongue English 

mother tongue French 

mother tongue other than English/French 


The choice of stratification variables was customized for 
each area that underwent optimal stratification. Within 
areas being optimally stratified, the above variables were 
obtained from the 1991 census. If a variable accounted 
for less than two percent of the total population, then it 
was dropped. For groups such as services, if subgroups 
such as financial services were not significant, then the 
grouped variable was used instead. A group was 
considered significant if it accounted for more than two 
percent of the population. 


Types of area for stratification. The LFS frame may be 
divided into three types of areas: (1) rural areas, (2) larger 
cities, (3) smaller urban areas. For stratification, each of 
these could be further divided, as described next. 


1. Rural areas. In rural areas, strata were usually formed 
by manually grouping two or three Census Divisions 
within an ER-EJER intersection into a geographical 
stratum. Decisions regarding the formation of these 
geographical strata were made in conjunction with 
decisions regarding the choice of first-stage sampling 
units that would be most appropriate (e.g., two-stage EA 
versus three-stage PSU design) and whether there should 
be separate rural and urban strata formed. Optimal 
stratification within the geographical strata was performed 
whenever there was sufficient population to warrant it. In 
general, the rural strata in the new design tend to be 
smaller than the rural strata in the previous design. 


2. Larger cities (population of 50,000 or more): In 17 
CMAs, there were enough apartment buildings to form a 
separate list frame, referred to as the apartment frame, 
described in 2.3 below. Excluding the apartment frame, 
the remainder of each large urban centre comprised an 
area frame. In addition, where feasible, areas with high 
average income also formed a separate stratum, described 
in 2.2. The remaining dwellings formed the regular strata 
and are described below under Street Network File (SNF) 
areas. As will be noted below in (2.1), SNF areas 
correspond to CADP (computer assisted districting 
program) areas. 


When more than one level of stratification exists in a city, 
we will refer to the lowest-level, smallest strata as final 
strata. Final strata were designed to have an expected 
sample size of at least 48 households (36 households in 
Toronto, Montreal and Vancouver). Expected sample 
sizes in Toronto, Montreal and Vancouver are smaller 
because in the new design, the sample yield per cluster 
was chosen to be about double the yield in the old design, 
and the latter yield was somewhat lower for these three 
cities. 


2.1 Larger Cities: SNF (or CADP) areas. These urban 
areas are covered by Geography Division’s Street 
Network File. They include the 25 CMAs and 20 of the 
largest Census Agglomerations (CAs). In these areas, 
there are up to three levels of stratification. Within a 
CMA or CA, if the anticipated sample size in a 
municipality (i.e., a Census Subdivision or CSD) within 
it was at least 240 households (180 households in 
Toronto, Montreal and Vancouver), then the municipality 
itself is a stratum. If the municipality is large enough to 


form more than five final strata, it is stratified optimally 
into groups, called superstrata, that will in turn yield three 
(sometimes four or five) final strata. Superstrata are 
formed using Census Tracts (CTs) as stratification units 
(or CSDs in non-tracted outskirts of cities). These 
superstrata are geographically compact and contiguous. In 
Toronto, Montreal and Vancouver, the aim was to create 
superstrata that would yield six final strata instead of 
three. 


If a CSD was not large enough to form more than five 
final strata, it was pooled with other such CSDs. This 
pool was then treated as a superstratum as in the previous 
paragraph, 1.e., if large enough, optimal strata were 
formed within it, and so on. 


Superstrata were divided optimally into the final strata; 
these are non-compact, non-contiguous strata yielding a 
sample of 48 households, except in Toronto, Montreal and 
Vancouver where they yield 36 households. 


2.2 Larger Cities: High income strata. For the first time, 
high income strata were formed in the nine cities where 
this was feasible, namely, in Montreal, Ottawa, Toronto, 
Hamilton, London, Winnipeg, Edmonton, Calgary and 
Vancouver. In each of these cities, the three percent of 
Enumeration Areas (EAs) with the highest average 
household income in the 1991 census formed the high 
income stratum. Each stratum had to be large enough to 
yield a sample of 24 households. In five cities there were 
enough EAs to form two or more high income strata. This 
is summarized in Table Al in Appendix A. In cities not 
on this list, there were not enough EAs with a high 
average household income (about $100,000) to form a 
separate stratum. More details on high income strata are 
given by Chen et al. (1994). 


The introduction of high income strata is expected to 
make the representation of high income households in the 
sample more stable over time. This will benefit surveys, 
such as the Survey of Consumer Finances, that are based 
on the LFS frame or sample and collect income-related 
data. It will also help in the collection of earnings 
information in the new LFS questionnaire. Another 
possible benefit is that it will be easier to investigate 
whether the propensity to be nonrespondents is different 
for high income households. If it is established that 
nonresponse is substantially higher in high income strata, 
special measures to address the problem can be 
developed. 


2.3 Larger Cities: The apartment frame. The LFS has 
maintained a list of apartment buildings in large CMAs 


since the 1960s. Currently, this list is used as a sampling 
frame in eighteen cities: Halifax, Quebec City, Montreal, 
Hull, Ottawa, Oshawa, Toronto, Hamilton, St. Catharines, 
Kitchener, London, Windsor, Winnipeg, Saskatoon, 
Calgary, Edmonton, Vancouver and Victoria. For the 
LFS, a building is treated as an apartment if it has at least 
five floors of living quarters and at least thirty residential 
units. In each city, as new buildings are constructed, they 
are added to the bottom of the list for that city. Since 
sampling of apartments is systematic, a new apartment 
building has a chance of falling in the sample as soon as 
it is built. 


A new feature in the LFS design is the formation of a 
frame of low income apartment buildings. In contrast to 
the high income strata, it was found more beneficial to use 
apartment buildings instead of EAs for stratification of 
low income households. 


An apartment building was added to the low income 
frame if its average household income according to the 
1991 census was less than $20,000. For a low income 
frame to exist in a city, the frame must contain enough 
dwellings to yield a sample of at least 30 dwellings and 
the average income for the whole frame should be about 
$15,000. Low income frames were created in seven cities: 
Montreal, Ottawa (excluding Hull), Toronto, Winnipeg, 
Calgary, Edmonton and Vancouver. 


Special cases: In Calgary and Edmonton, buildings with 
average incomes greater than $20,000 were added to the 
frame to bring the sample yield to the desired level. In 
Montreal, the low income frame exists entirely within ER 
40. 


Unlike the rest of the apartment frame, the low income 
frame is not open-ended since the average income of the 
residents of a newly-constructed apartment is not 
available. 


Of the seven cities with a low income frame, only Toronto 
has enough units to support stratification of the frame. 
The CSD of Toronto and the rest of the CMA of Toronto 
each have a low income apartment frame. 


For the non-low income apartments, an attempt was made 
to create list frames within geographic superstrata. This 
was possible in Halifax (2 apartment strata), Quebec (2), 
Montreal (4), Ottawa (3), Toronto (6), Hamilton (2), 
Kitchener (2) and Vancouver (4). These amount to 
geographical breakdowns of the overall apartment strata 
within these cities. 


Finally, an attempt was made to further subdivide 
apartment strata according to apartment size. In each 
stratum, apartment buildings were classified according to 
size: less than 100 units, between 100 and 199 units, and 
200 units or more. If there were enough apartments in a 
size class to yield a sample of thirty dwellings, it became 
a separate stratum. Otherwise it was collapsed with 
another class. 


The apartment frame stratification is summarized in Table 
LE 


Table 1. Apartment Frame Strata 


Total Number 
of Strata 


Geographical 
Strata 


Note: (i) A * denotes that the city has low income strata. 
(11) The total number of strata includes low income strata. 


3.1 Smaller urban areas: EA design (see Appendix D). In 
all cities except the smallest ones, non-compact, non- 


contiguous optimal final strata were formed using EAs as 
stratification units. In Sydney, Nova Scotia, first compact, 
contiguous superstrata were created, and then the final 
strata were formed within each superstratum. 


3.2 Smaller urban areas: VR design (see Appendix D). In 
the smallest urban areas, if the area was classified as "self- 
representing" in the old design (i.e., it constituted at least 
one urban stratum with a sample size of at least 50 
dwellings), its old stratification was used in the new 
design. In some cases, additions to the urban area were 
assigned manually to a stratum. The term VR design is 
used here because at the time of the previous redesign, 
counts from census visitation records (VRs) were used to 
form clusters. 


Clustering in final strata. To reduce field costs, 
households in final strata are not selected directly. 
Instead each stratum is divided into clusters, and then a 
sample of clusters is selected in the stratum. Then, in each 
selected cluster, a sample of households is chosen. The 
methods used to select clusters and households are 
discussed later in the section on sample selection. 


In rural areas, EAs are usually used as clusters. In urban 
areas, a variety of clusters are used. In the smallest urban 
areas, where the stratification from the previous design 
was usually retained, the old clusters were also used. 
These were formed manually using census visitation 
records: blockfaces were combined until the desired 
cluster size was achieved. The population counts for 
these clusters were updated using 1991 census 
information, and in some cases, the old clusters had to be 
modified because there had been major changes since the 
old design. In urban areas where stratification was based 
on EAs, EAs were also used as clusters. In some cases, 
large EAs were split into two clusters. 


In the largest urban areas, i.e., those with a SNF, a new 
automated clustering method was used, replacing the 
labour-intensive manual operation used in the past. For 
the new design, the CADP program that was used to form 
EAs for the 1991 Census was modified by staff in 
Statistics Canada’s Geography Division to form LFS 
clusters. The program combines block faces to produce 
clusters containing 150 to 200 dwellings (200 to 250 
dwellings in Montreal, Toronto and Vancouver), on 
average. 


The new urban clusters are about three times the size of 
clusters used in the previous design. The increased cluster 
size will help attenuate problems due to the rapid 


population growth that occurs at times in urban areas, 
since the relative impact of growth will tend to be smaller 
for large clusters. The larger cluster size also reduces the 
frequency of cluster rotation. 


Table 2 gives a summary of the types of first stage units 
used for the bulk of the LFS sample. The size refers to 
the number of households in a typical unit and can vary 
widely for a specific type of unit. The yield is the number 
of households selected by the LFS for interviewing in any 
given month. 


Table 2. Major first-stage units, sizes and yields. 


Area Sampling Size Yield 
Unit (households (sampled 
per unit) households) 


Toronto, Montreal, Vancouver 200-250 


Most rural areas and EA 300 
non-SNF parts of cities 


Appendix D lists all urban areas in Canada that are LFS 
strata or groups of strata, i.e., it lists all urban areas that 
always contain some LFS sample. This corresponds to the 
old self-representing unit concept in previous LFS 
designs. Any urban areas not listed in Appendix D are not 
strata. They are either primary sampling units, or parts of 
a larger urban stratum, or are incorporated into a rural 
stratum. Appendix D also indicates the types of clusters 
used in each area. 


CHAPTER 3: Sample Allocation, Selection and Rotation 


During the redesign, the total size of the monthly Labour 
Force Survey sample was kept at the level that existed in 
the old design, namely, 58,850 households. As part of the 
redesign process, this sample was allocated to best meet 
the need for good estimates at various geographical levels. 
These include the national, and provincial levels, Census 
Metropolitan Areas, and for the first ttme, Unemployment 
Insurance regions, which are now called Employment 
Insurance Economic Regions. The following were the 
reliability targets used. 


Canada and the provinces: Maintain or improve CV levels 
for unemployed from the old design, 1.e., a CV of about 2 
percent for Canada and 4 to 7 percent for provinces. 


EIERs/CMAs: CVs of 15 percent or less for quarterly 
estimates of unemployed. A minimum sample size per 
EIER of 600 households per month was set. 


Though not a formal requirement, a target CV of 25 
percent or less for quarterly estimates of unemployed for 
ERs was also used, although some collapsing of regions 
was necessary. There are 72 ERs, but for allocation, this 
number was reduced to 68 by collapsing a pair of ERs in 
each of Quebec, Manitoba, Saskatchewan and British 
Columbia. The CV target is for the collapsed regions. 
Table 3 summarizes the number of subprovincial regions 
in each province at the time of the redesign. 


Table 3. ERs, EIERs and CMAs by province. 


| Prince EdwardI. | 1 {| 1 | 0 | 


*The Ottawa-Hull CMA is counted in both Ontario and Quebec. 


As was noted in the stratification section, ER-EIJER 
intersections were used as strata. As a result, they were 
also the basic areas used during sample allocation. 
Because both ERs and EJERs tend to use census divisions 


as building blocks, there are only 133 intersections 
throughout Canada. 


Allocation of the sample to provinces and to regions 
within provinces was discussed with the provinces as well 
as major users of LFS data. The final allocation was also 
affected by operational constraints. Several allocation 
strategies were studied, including Neyman, Kish, 
proportional, power and square root allocation. They are 
summarized by Mian and Laniel (1994). Here, only the 
approach actually implemented is described. 


At the time of the redesign, the total LFS sample size 
consisted of a core sample of 42,310 households and a 
HRDC-funded sample of 16,500 households. Because of 
changes in the sample size and in the population during 
the years since the previous redesign, the allocation of the 
core sample was optimal for neither provincial nor 
subprovincial estimates. 


The overall strategy was to first allocate the core sample 
to optimize provincial and national estimates. This was 
followed by allocation of the HRDC sample to 
supplement the core sample in the EJERs that need it most 
(typically, these will be regions with relatively small 
populations). 


Allocation of core sample to provinces. With the 
following exceptions, the core provincial sample sizes that 
existed before the HRDC-funded increase were retained. 
The exceptions are: a transfer of sample from 
Saskatchewan to Manitoba and from Alberta to BC. The 
amount of sample transferred was just enough to give 
each pair of provinces equal CVs for unemployed based 
on the core sample. This was done to correct imbalances 
in the old allocation that were introduced via sample 
decreases that occurred in the life of the old design 
(provinces with larger CMAs were harder hit by decreases 
in sample since the cuts tended to be concentrated in large 
CMAs). 


Allocation of core sample to ERs within provinces. For 
each province, the core sample was allocated to ERs in 
proportion to the size of these regions, where size is 
measured as the number of private occupied dwellings in 
the ER according to the 1991 census. This is an 
allocation primarily aimed at optimizing estimates at the 
provincial level. However, since sparsely populated ERs 


receive too little sample if proportional allocation is 
followed strictly, a minimum sample size was set, namely 
200 households per ER. In Alberta, where the minimum 
was set at 300 households, some efficiency at the 
provincial level was sacrificed to benefit small ERs. 


Allocation within ERs. The samples allocated in the 
previous step were then allocated proportionally within 
each ER to ER-EIER intersections. Again, this is a nearly 
optimal allocation of the sample. 


Allocation of the HRDC-funded sample to EIERs. The 
previous step completed the allocation of the core sample. 
The next step was to allocate the HRDC sample of 16,500 
households. Since HRDC funds these extra households to 
ensure that labour force estimates for EIERs are of 
adequate quality, the sample was allocated to EIERs to 
maximize the improvement in CVs of unemployed, 
targeting the sample to EIERs that had high CVs based on 
the core sample alone. Allocation of the sample in this 
fashion resulted in a CV of 10 percent or better for the 
estimate of unemployed in each EIER. A minimum sample 
size of 600 households was allocated to each EJER. 


Table A2 in Appendix A gives the core and total 
allocations by province for the old and the new design at 
the time of the redesign, as well as the allocation after the 
sample reduction that took place in 1995. 


Calculation of CVs. At various stages in the allocation 
process, CVs for the characteristic unemployed had to be 
calculated. The variance of this characteristic is a 
function of the unemployment rate. The average 
unemployment rates from the LFS for provincial and 
subprovincial areas for the period from 1984 to 1992 were 
used. This period was chosen to reflect unemployment 
rates typical of those likely to be encountered during the 
life of the survey design. In addition, since the LFS design 
involves clustering, the variance can be expressed as a 
function of a design effect as well as the unemployment 
rate. [The design effect of an estimator is the ratio of its 
variance under the actual design to the variance it would 
have under a simple random sample of the same size.] The 
design effects used were based on estimated design effects 
from 1989 to 1992---smoothed averages of design effects 
for each ER were calculated. A more detailed explanation 
of how CVs were calculated is given by Mian and Laniel 
(1994). 


Sample size reduction. Following the implementation of 
the new design, the size of the core sample was decreased 
by 6500 households. The reduction took effect in July 
1995. Because of the more efficient design, the CVs of 


national and provincial estimates after the decrease are 
comparable to the CVs that existed prior to the redesign 
with the larger sample. The sample was reduced by the 
same percentage in each province. The current sample 
sizes for each ER and EIER are given in Table A3 in 
Appendix A. 


Sample Selection 


Stages of sampling. One of the major changes in the new 
LFS design is the use of only two stages of sampling in 
almost all areas. The first stage is an area sample. For first 
stage units that are selected, a list of dwellings is prepared 
(and maintained) in the field. A sample of dwellings is 
then selected at the second stage from each list. 


The replacement of three stages of sampling in rural areas 
under the old design by a two-stage design has several 
benefits. In addition to being simpler, the two-stage 
design is more statistically efficient than the old design. 
Since the location of first-stage sampling units is now less 
constrained, the spread of the sample is improved. This 
can be an advantage for small area estimation; see Singh 
et al. (1994). 


Rural areas. In the new design, within rural final strata, 
EAs are selected at the first stage of sampling followed by 
the selection of dwellings at the second stage. The EAs 
are selected using randomized PPS systematic (rppss) 
sampling, which is described in the next paragraph. PPS 
denotes probability proportional to size, and here, the size 
of a sampling unit is the number of households in the unit 
during the 1991 Census. Within selected EAs, a 
systematic sample of dwellings is chosen at random. 
Usually, ten dwellings are chosen in this way in each EA. 
In the old design, three stages of sampling were used in 
most rural areas, where primary sampling units (PSUs) 
consisting of groups of EAs were selected first. Next, EAs 
were selected within PSU, followed by dwellings within 
EA. This three-stage approach was most useful when most 
interviews were conducted in person since the PSU 
corresponded roughly to one interviewer’s assignment and 
was convenient for travel. Since five-sixths of interviews 
are now conducted by telephone, a simpler design is 
feasible in most parts of the country, and the old PSU 
stage has been eliminated except in some remote areas. A 
comparison of design alternatives for rural areas that was 
done in the course of the current redesign project is given 
by Mantel et al. (1994). 


The randomized PPS systematic method of sample 
selection, developed by Hartley and Rao (1962), has been 


used in the LFS since the 1970s. In this method, the first 
stage units in a stratum are put in random order and then 
a PPS systematic sample of the desired size is selected. In 
the new design, the first stage unit is the EA, and six EAs 
are selected per stratum. PPS systematic sampling is 
described by Cochran (1977). 


In the rural areas with the lowest population density, a 
different design was used. Geographically compact first 
stage units consisting of six EAs were formed and two or 
three such first-stage units were selected using 
randomized PPS systematic sampling. In the selected first 
stage units, a systematic sample of dwellings was selected 
in each of the six EAs, i.e., no sub-sampling of EAs took 
place. This clustered design was adopted to ensure that 
there would be sufficient work to occupy an interviewer 
in sparsely populated areas that fell in the sample. 


Major urban areas, non-apartment frame. In urban areas, 
the first stage involves the selection of clusters. Since the 
Street Network File does not provide complete coverage 
of cities, particularly their outskirts, clusters could not be 
formed everywhere, and EAs or parts of EAs were used as 
first stage sampling units. At the second stage, a 
systematic sample of dwellings was selected within 
cluster. In the apartment frame, the apartment building 
plays the role of a cluster. Thus in both urban and rural 
areas, a two-stage design is the norm. 


The selection of clusters and EAs in urban areas is done 
using the random group method due to Rao, Hartley and 
Cochran (1962); see also Cochran (1977). This method 
was introduced in the 1970s because it is amenable to the 
relatively straightforward revision of cluster selection 
probabilities. Such revisions may occur between major 
redesigns in parts of cities that have undergone rapid 
population growth. The method is also flexible for 
dealing with the changes in the LFS sample size that take 
place from time to time. We now present an overview of 
how the Rao-Hartley-Cochran (RHC) random group 
method is implemented in the LFS. Additional details can 
be found in Singh et al. (1990). 


For a stratum in which the RHC method is used, the 
clusters are assigned at random to six groups called 
random groups. The number of clusters in each random 
group is made as equal as possible, i.e., it varies by at 
most one cluster. In some cases, a multiple of six groups 
is used. In each random group, one cluster is selected 
with probability proportional to size; e.g., if a cluster is 
twice as big as a second cluster, then the first cluster has 
twice the probability of being selected as the second 
cluster. 
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Within selected urban clusters in non-high income strata, 
a systematic sample of dwellings is selected. In Montreal, 
Toronto and Vancouver, six dwellings per cluster are 
selected. In other urban areas, eight dwellings per cluster 
are selected. For clusters in high income strata, a 
systematic sample of four dwellings is selected. 


Major urban areas, apartment frame. In each apartment 
frame stratum, which is an open-ended list of buildings, 
apartment buildings are selected using PPS systematic 
sampling (randomized PPS systematic sampling is used in 
the low income strata). Within each selected apartment 
building, a systematic sample of five dwellings is selected. 


Other urban areas. In almost all other urban areas, in the 
first stage, either clusters or EAs are selected using the 
RHC random group method, followed by the selection of 
dwellings. The number of dwellings selected per first 
stage unit varies, from three (for clusters) to ten (for EAs), 
because this design covers a broad range of urban and 
semi-urban areas. 


In a few special cases, accounting for less than one 
percent of the sample, the first stage of sampling is to 
select two towns in a stratum using randomized PPS 
systematic sampling. Then a multiple of six (usually 12 
or 18) clusters are selected in each town using randomized 
PPS systematic sampling. Finally, a systematic sample of 
three dwellings is selected in each cluster. Note that here, 
"town" can refer to an actual town, two small towns 
treated as one, or part of a larger town. This design was 
used in cases where it was not practical to obtain 
reasonable interviewer assignments with the other designs 
discussed above. 


Remote areas. In the seven non-Maritime provinces, most 
of the northern part of each province is sparsely 
populated. As a result, the LFS uses a special design for 
these areas. With one exception discussed later, the 
sample is selected in two stages. The first stage consists of 
a sample of settlements, which we refer to as places, and 
EAs. Because of the long distances involved when 
interviewing in remote areas, places with fewer than 10 
households or 25 persons are omitted from the design. 
Similarly, EAs with fewer than 25 households are omitted. 
Despite these omissions, the design covers about ninety 
percent of the remote population in each province. 


A sample of EAs and places is selected using systematic 
PPS sampling after the units are sorted by number of 
households. Then a sample of dwellings is selected using 
systematic sampling. If a selected place or EA is too big 


to be listed conveniently, it will be split into manageable 
clusters. 


Quebec has two remote strata. One stratum follows the 
above design and the other follows a three stage design. 
The latter stratum contains eight towns which form the 
first stage units. The second stage units are clusters 
obtained from EAs by splitting large EAs and collapsing 
small EAs. The target cluster size is 100 households, with 
a tolerance of about fifty households. Two towns are 
selected using randomized PPS systematic sampling. Then 
three clusters per town are selected using the same 
method, and finally, nine households are selected 
systematically in each cluster. 


Sample Rotation 


Each month, a portion of the LFS sample is replaced. 
Rotation of sampling units occurs at each stage of the 
multi-stage sample design. The ultimate unit of selection, 
the dwelling, is replaced every six months, whereas 
higher-level units remain in the sample for longer periods 
of time. The determination of six months as the period for 
rotation of households is a trade-off between the cost of 
rotation and the increase in nonresponse that might occur 
if respondents were asked to remain in the survey for a 
longer period of time. 


To ensure uniform interviewer workloads and to minimize 
the effect of any bias due to the number of months a 
dwelling has been in the survey, a rotation scheme was 
adopted whereby one sixth of the dwellings rotate each 
month. This is achieved by associating with each cluster 
a rotation number between one and six. This number 
determines the months in which the rotation of households 
(their birth months) take place: If the rotation number is 1, 
then dwellings in the cluster rotate in January and July, if 
2, then in February and August, and so on. 


Method of Rotation 


Areas using the random group method: This cluster 
selection method was described near the end of the 
previous chapter. For the initially selected cluster within 
each random group, two numbers between one and the 
cluster inverse sampling ratio (ISR) are generated at 
random. The first determines a random start for systematic 
selection of dwellings within the cluster. The second 
determines the number of systematic samples of dwellings 
to be drawn from the cluster, that is, the number of six- 
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month periods for which the cluster will remain in the 
sample. 


Prior to each occasion for selecting a new sample of 
dwellings, the random start for the cluster is incremented 
by one, until the incremented value would exceed the 
cluster ISR, at which time the start reverts to 1. Cluster 
rotation occurs at the end of the randomly determined 
number of sampling occasions. 


The random retention period for initially selected clusters 
is necessary to ensure that initial probabilities of selection 
of units are preserved over time. If, for example, initially 
selected units were retained until exhausted (that is, until 
all systematic samples of dwellings were used), this would 
eventually result in a sample with overrepresentation of 
larger units. 


Cluster rotation is carried out by proceeding to the next 
cluster on the randomized list of clusters in the group. If 
the cluster rotation proceeds to the end of the list, the 
selection reverts to the first cluster on the list. As with 
initially selected clusters, the selection of dwellings is 
governed by a random start between | and the ISR which 
advances at each sampling occasion. 


Areas using randomized PPS systematic sampling: 
Rotation of dwellings and clusters proceeds as described 
in the previous paragraph. There are a few urban strata 
with 3-stage sampling. The first stage selects PSUs within 
strata while the second stage selects clusters within PSUs 
and the last stage selects dwellings within clusters as 
usual. The same sample rotation applies to each stage of 
sampling. Urban PSUs can rotate within strata and 
clusters rotate within urban PSUs. 


Replacements for initially selected units, and subsequent 
replacements, all remain in the sample until exhausted 
subject to the minimum life rule. The minimum life rule 
attempts to delay the onslaught of cluster rotation after the 
initial selection. This is most important in rural areas 
where rotation may require hiring new interviewers, but 
the concept was applied to all areas. The bias induced by 
lengthening the life of the initial selection is negated by 
shortening the life of the subsequent selection. As a result, 
the second unit stays in the sample until its regularly 
scheduled rotate-out month as if the minimum life 
adjustment had not taken place. Subsequent selections 
remain active in the sample for full life. 


The following equation must be satisfied for unbiased 
selection probabilities: 


Here K, is the minimum number of starts for the initially 
selected unit, K, is the minimum number of starts for the 
subsequent replacement unit and R_., is the smallest ISR 
of all the units in the stratum. 


The value of K, is selected based on the following rules, 
starting with a base value of b. 


If b< (R,,,41)/2 then K, isia 
random number in the range 
(bi Sheeh ells 


If b > (R,,,,+1)/2 then 
K, =int(R,;,/2)*1. 


An initial selection with a life r, that is less than K, will 
be extended by K,-r,. The subsequent selection will be 
shortened in life by the same value of K,-r,. It follows 
thataK, +Kjos sR is 


Based on empirical evidence at the time of design, most 
strata were optimized with a basic minimum of four starts 
(two years in sample) whereas the apartment frame was 
optimized with a base of two starts. Since the apartment 
frame is an open-ended frame, the minimum life rule is 
applied to new selections as well, but in this instance a 
base of 4 is used. The choice is more for convenience to 
avoid new cluster listing for very short life spans. 


Operational Steps in Sample Rotation. Rotation of the 
sample is automated within a system known as the Sample 
Design System, which identifies the units that are rotating 
in and out of the sample for each survey. These rotating 
units require manual processing by Sample Control staff, 
who identify the geographical area represented, and form 
lower stage sampling units where applicable. 


Each different type of frame in the sample design uses 
cluster selection programs that create rotation records. 
These rotation records describe the sequence of advancing 
starts within clusters and the sequence of between cluster 
rotations corresponding to each initial selection within a 
group. Each record lasts the life of the group, up to a 
maximum of 40 random starts. The collection of these 
records constitutes the master rotation file, which is used 
to automatically rotate the sample. The master rotation file 
is changed as rotation patterns are updated (as in the open- 
ended apartment frame), and as old records become 
exhausted and are replaced by new ones. 


The way in which the rotation of households is 
accomplished is as follows. Seven months before a 
particular survey date the design information on all 
clusters rotating for that date is identified. This includes 
new and existing clusters. 


For the new clusters, Sample Control maps out cluster 
diagrams (FO1s) as required. The completed FO1s are sent 
to the Regional Offices (ROs) 20 to 23 weeks before the 
date of introduction in order to initiate the field listing. 
The interviewers’ laptop computers are also sent new- 
listing control files to match these FO1s. The dwellings 
captured by the interviewer are transmitted back to the 
central database. 


The central database has already been fed with the 
random start and inverse sampling ratio to be applied to 
each cluster, new and existing, in the sample for a 
particular survey and date. A selection of households is 
accomplished by a systematic sample of the lists about six 
weeks prior to the interview week. Household level 
records are sent back to the interviewers’ laptops for 
interviewing. These selected households remain in the 
sample for six months. 


Every six months another set of starts for these clusters is 
sent to the database, thereby resulting in selection of new 
households. In the meantime starts for other rotations 
have been selected as well. 


In the three-stage sample design areas, a rotation record 
lasts only as long as a PSU remains in the sample. Thus, 
as records become exhausted, it can be determined which 
PSUs need replacing. By referring to the design frame, 
replacement PSUs are determined. In the PSUs identified 
as rotating in, Sample Control determines their 
geographical location, and prepares maps of the area. 


Ordinarily, the process of PSU rotation begins at least 30 
weeks before the date of introduction of the sample. The 
advanced schedule is required since not all the clusters 
have been formed in these new areas. With the rotation of 
PSUs, the subsequent stages of selection, namely clusters, 
will have to be carried out, generating additional rotation 
records 


Assigning Rotation Numbers. In assigning rotation 
numbers, the objective is to evenly distribute the expected 
sample take. The expected take is just the yield from 
sampling all clusters based on the design count of 
households used in creating the frame. This distribution is 
achieved simultaneously for the sample as a whole, and at 
the same time for the smallest possible geographical 
subsets of it. Adherence to these objectives implies the 
following. 


¢ The workload of interviewers is stable, as roughly 
equal numbers of units are rotating each month. 


¢ The sample is comprised of equal numbers of 
households having been in the sample for 1 vs. 2 ... 
vs. 6 occasions, nullifying time-in-sample effects as 
a cause of differences in estimates between areas, 
or over time. 


¢ The sample is effectively divided into six equally 
representative parts, which may be used when sub- 
samples from the LFS frame are desired. 


To achieve the aforementioned objectives, the initially 
selected clusters were assigned rotation numbers in such 
a manner as to balance the total expected take within each 
stratum and EJER. The assignment of rotation numbers 
was accomplished independently within each EIJER, but a 
random starting factor was built in to distribute the 
expected take as evenly as possible at higher levels. 


In most areas every stratum has six or a multiple of six 
selections so that each rotation group can have the same 
number of selections. In rural three-stage designs, 
anywhere from six to nine clusters are formed, leading to 
some stratum level collapses to form 6 rotation groups. In 
general the smallest units were combined to create a more 
or less even distribution. The apartment frame has a 
variable number of selections that are assigned rotation 
numbers randomly. The remote frame typically had less 
than 6 selections and were left out of the general picture - 
their actual yields are very uncertain in any case. 


Due to the differences between the size and numbers of 
sampled units per stratum in apartment, urban and rural 
areas, the first step in balancing the sample take by 
rotation was to balance it as much as possible for the 
larger rural units at the EIER level. Rotation numbers 
were then assigned to the highly variable apartment units 
and then to the urban units to balance the take at the 
regional level. 


Basically the assignment takes place with a sorted array of 
expected takes by rotation. For each stratum in the list of 
one EIER the takes by rotation are sorted from minimum 
to maximum. A running count of takes collected so far is 
similarly sorted in reverse order. The rotations are 
assigned by matching the rotation with the minimum take 
in the stratum to the rotation with the maximum value in 
the running count. Initially the rotations are given a small 
random take. At the end of the list the takes should not 
vary by more than the variation within any one stratum. 
Note that these are design takes, whereas actual sampling 
will vary considerably from this expected value in some 
cases. 


Being open-ended, the apartment frame requires rotation 
assignment on a continuing basis. Every set of six 
selections in the systematic sample was assigned the six 
rotation numbers in a random order. No balancing is 
possible in this open-ended frame. For new additions to 
the list, a set of six randomly ordered rotation numbers is 
generated as required. The assignment of rotation 
numbers would entail choosing the next available rotation 
from this set. 


Similarly in strata with three-stage designs, the cluster 
selection stage is not complete until the PSU rotates into 
the sample. The new PSU must be assigned rotation 
numbers at the time of introduction. A residue of design 
takes within the region is compiled without the rotating 
sample in question. Then the same technique is used to 
assign rotations to the new PSU. 


Occasionally a cluster is assigned a rotation that does not 
follow the standard date of introduction. This is called an 
off-rotation selection. The prime candidate is the 
apartment frame. Being open-ended, a new cluster 
selection can occur in any month. At the same time the 
rotation assigned is random. Rather than wait for the 2-5 
months to introduce the sample on rotation, it is 
preferable to send the cluster out as quickly as possible 
for interviewing, implying off-rotation. 


Changes to the Labour Force Survey after the 
Redesign 


After the introduction of the new LFS sample was 
completed in March 1995, there were two major changes 
to the survey. The first was the reduction by 11 percent of 
the sample beginning in July 1995. Because the new LFS 
design is more efficient than the previous one, the CVs of 
national and provincial estimates after the decrease are 
comparable to the CVs that existed prior to the redesign 
with the larger sample. The sample was reduced by the 
same percentage in each province. 


The second major change was the redefinition of the 
Unemployment Insurance Regions by the Human 
Resources Development department. In 1996, the 61 
UIRs were replaced by 53 Employment Insurance 
Economic Regions. Once the boundaries of the new 
regions were finalized, the sample size per region, and the 
resulting quality of LFS estimates, were studied. In five 
cases, the sample allocated to a region was too low to 
meet the requirements of the Employment Insurance 
program. As a result, the sample in these regions was 
increased in 1997, with a corresponding decrease in the 
sample size in regions which could now afford a 


reduction. At the national level, the LFS sample size 
remained the same, but there were some shifts in sample 
among provinces. Table AO in Appendix A presents an 
overview of the LFS after the above changes took place. 


CHAPTER 4: Special Surveys and Supplementary Surveys 


Many household surveys use the 
Labour Force Survey frame and 
sample to collect information. Surveys 
that do this by interviewing 
households that have also been 
selected for the LFS are referred to as 
supplementary surveys. Surveys that 
use the LES frame to select a different 
sample of households are referred to 
as special surveys. For _ special 
surveys, the households are usually 
selected in clusters that are also being 
used for LFS interviews. Use of the 
LFS frame and sample in this way 
results in substantial cost savings for 


Each of the six rotation groups of the LFS can be used to 
produce estimates. Typically, special and supplementary 
surveys use from one to five rotation groups for their 
sample, depending on the required level of reliability. 
Usually, the LFS birth rotation group, i.e., the one 
consisting of households being interviewed by the LFS for 
the first time, is avoided because the initial LFS interview 
takes longer to complete than subsequent interviews. 


these _—_ surveys. Special and 
supplementary surveys are often 
sponsored by other government 


departments. Note that supplementary 
surveys can be divided into two types: 
those that use LFS households while 
they are still being interviewed for the 
LES and those that are no longer being 
interviewed by the LFS, sometimes 
referred to as rotate-outs. 


Examples of Major Special and Supplementary 
Surveys 


The Survey of Consumer Finances (SCF) is an annual 
household survey conducted in April. It is a supplement 
to the LFS based on all households in four rotation 
groups. Each household is sent a questionnaire by mail 
prior to the April LFS interview. The information from 
the household is then collected during the LFS interview 
using computer assisted interviewing. The major outputs 
of the SCF include income distributions before and after 
taxes, and mean and median incomes.These results have 


In some cases, only part of a rotation group’s households 
are required. To achieve this, dwellings are dropped at 
random as in the LFS stabilization program. Selection can 
also take place within households by either random 
sampling or by screening for individuals with specific 


characteristics. 


The following table lists some of the surveys using LFS 
rotations or the LFS frame in 1998. 


19 


been used to derive income-related measures such as 
Low-Income Cut-Offs. Plans are underway to integrate 
the SCF with the Survey of Labour and Income 
Dynamics, discussed below. 


The Survey of Household Spending(SHS) is a new annual 
household survey that replaces both the Family 


Expenditure Survey (FAMEX) and the Food Expenditure 
Survey. The new survey is being introduced as part of the 
Program to Improve Provincial Estimates (PIPES). The 
overall objective of PIPES is to produce more reliable 
provincial estimates for use in the tax allocation formula 
for the Harmonized Sales Tax. The Survey of Household 
Spending will also continue to be used in its traditional 
role as a source of information for computation of the 
Consumer Price Index. The SHS is a special survey, i.e., 
it selects households in clusters containing LFS sample, 
but the SHS households are not interviewed by the LFS. 


The new survey will be very different from FAMEX. The 
latter was conducted every four years, but the SHS will be 
conducted annually. In addition, the SHS’s sample will be 
almost double that of FAMEX. As a result, the new survey 
will deplete available LFS clusters more quickly. The 
biggest difference between the surveys will be in the data 
collection methodology. The long, detailed FAMEX 
questionnaire will be replaced by a more streamlined, 
mixed-mode sequence of contacts with responding 
households. 


Longitudinal Surveys. In the 1990s, Statistics Canada 
developed several new longitudinal surveys to obtain data 
that would fill certain information gaps about Canadians. 
The major new surveys, which all used the LFS for 
sample selection, are the Survey of Labour and Income 
Dynamics, the National Longitudinal Survey of Children 
and Youth and the National Population Health Survey. 
We will now describe these surveys briefly. 


The Survey of Labour and Income Dynamics 
(SLID) was introduced to study the processes that 
influence the economic life of Canadians. The 
survey is used to investigate movements into and 
out of low income status, labour markets transitions 
and the relationship between family dynamics and 
economic well-being. The first panel of the Survey 
of Labour and Income Dynamics was introduced in 
1993, followed by the second panel in 1996. The 
two panels overlap, and each panel is in the survey 
for six years. Thus the first panel will be replaced 
by a new one in 1999. Each panel initially consists 
of households that were recently interviewed by the 
LFS. During the life of a panel, individuals in the 
panel are interviewed up to twelve times, 
alternating between interviews about labour status 
in January and about income in May (persons can 
avoid the May interview by giving permission to 
Statistics Canada to used their administrative 
income tax data). Like other longitudinal surveys, 
SLID follows sampled individuals over time, even 
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if they move to another province or out of the 
country. 


Since SLID will be used to produce cross-sectional 
estimates as well as longitudinal ones, the sample 
in the two ongoing panels will be supplemented 
annually by a top-up sample to ensure that the total 
sample is representative of the population at a 
point in time. Each top-up sample will consist of 
10,000 households that will be combined with the 
longitudinal sample (30,000 households) to 
produce cross-sectional estimates. 


The National Population Health Survey (NPHS) 
was introduced to measure the health of Canadians 
over time. The NPHS was the first survey to use 
the new LFS design, beginning in June 1994, when 
it selected 25,000 households. These households 
were not interviewed by the LFS. To meet special 
demands, including sample buy-ins by some 
provincial governments, the NPHS supplements 
the sample selected from the LFS frame by 
samples selected using random digit dialing 
(RDD). These yielded an additional 800 
households in the first wave of the survey. In the 
second wave, approximately 60,000 additional 
households were surveyed using RDD to produce 
sub-provincial data for three provinces. 


To take seasonal factors into account, the NPHS 


sample is distributed over four quarters, with 
interviewing taking place in February, June, 
August and November. 


The NPHS selects one member of each initially 
selected household for an in-depth interview and 
follows him or her over time. Interviews are 
conducted every two years, for a planned duration 
of 20 years. At each wave, for cross-sectional 
estimation purposes, basic health information is 
collected for all members of the household 
currently residing with the longitudinal respondent. 


A more detailed description of the methodology of 
the NPHS is given by Tambay and Catlin (1995). 


The National Longitudinal Survey of Children and 
Youth (NLSCY), which began in 1994, tracks a 
sample of children over many years to monitor 
their development from infancy to adulthood. It is 
a complex survey which began with LFS-based 
households to obtain a sample of children, and then 
obtained information from children’s teachers and 
principals. Because only about 30 per cent of LFS 
households had children in the appropriate age 


range, it was necessary to use more than six LFS 
rotation groups to attain the desired sample. In 
most cases, eight or nine rotation groups were 
used. In addition to the sample obtained directly 
from the LFS, the NLSCY also includes children 
from 2500 households selected in the first wave of 
the NPHS. The direct LFS sample and the NPHS 
sample included approximately 21,000 and 4000 
children under the age of 12, respectively, for a 
total initial sample of 25,000 children. The 
sample, which will be contacted every two years, 
will be augmented each time with children in age 
groups not represented by the initial sample. 


The NLSCY uses a variety of questionnaires and 
collection methods. The initial household interview 
was conducted in person using computer assisted 
interviewing. Ten and eleven year old children 
completed a self-administered questionnaire. Each 
child’s teacher and principal were identified and 
asked to complete a questionnaire. For the latter, a 
mail-out/mail-back approach was used. In addition 
to the usual types of questionnaire, the NLSCY 
also administers tests to children, namely a 
mathematics test and a test of receptive vocabulary. 
For more information on the NLSCY, see Brodeur 
et al. (1995). 
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CHAPTER 5: Weighting and Estimation 


Introduction 


Estimates are obtained from the sample data using 
knowledge of the sample design and by employing 
estimation techniques from the theory of survey sampling. 
Each person in the sample receives a weight which we 
will refer to as the final weight. This weight represents the 
respondent’s contribution to the total population and is 
used to derive estimates for all characteristics of interest. 
This weight is derived as the product of three factors: a 
design weight, which incorporates design information; a 
nonresponse adjustment, which compensates for 
nonresponding households; and a factor (the g-factor) that 
calibrates the sample to known population counts. 


Once the estimates are derived, it is necessary to judge 
their reliability. Because the LFS is a probability sample 
it is possible to estimate the sampling error associated 
with each estimate. The sampling error can be used to 
make probability-based statements about survey estimates. 


Occasionally, estimates are required for regions that were 
not planned for at the time of design of the survey or 
whose boundaries changed after the design of the survey. 
This is the case for Employment Insurance Economic 
Regions (EIERs). Estimates for these regions are required 
by Human Resources Development Canada (HRDC) to 
administer the Canada Employment Insurance Program. 
By using small area estimation techniques, improvements 
in quality can be obtained for such regions. 


The purpose of this chapter is to describe the 
methodology used by the LFS to derive estimates and to 
provide the rationale behind the methods employed. This 
is followed by a description of the method of estimating 
the sampling error. A section is devoted to the method of 
deriving estimates for EJERs. Finally, changes to the 
estimation methodology between the present and previous 
sample designs, and the auxiliary information used in 
weighting are discussed. 


The Design Weight 


In any sample survey a target population is defined. The 
target population is the subset of the population that the 
characteristics of interest refer to. In any given sample, 
some members of the target population are selected and 
others are not. The selected members can be thought of as 
representing the non-selected members. In a probability 
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sample, each member has a known probability of being 
selected. If that selection probability is one in fifty, say, 
then the member represents 50 persons in total. One could 
make 50 copies of the survey responses and by repeating 
this procedure for every member in the sample, create a 
“pseudo-population”. This pseudo-population could be 
used for deriving the required estimates, since if the 
sample is representative of the population, then 
tabulations carried out on the pseudo-population will be 
very close to what would have been obtained had the true 
population been used. In practice, the records are not 
duplicated but rather are assigned a weight. Since this 
weight is determined by the sample design, it is referred 
to as the design weight. The design weight can be thought 
of as the number of times the record would have been 
replicated. 


For the LFS, the following facts affect the details of the 
estimation procedure. 


1) The survey uses a stratified, multi-stage design, with 
sampling conducted using probability proportional to 
size (PPS) selection at all stages except the final stage 
which uses systematic sampling. 


2) Since the ultimate sampling units are households, the 
design weights in the LFS refer to households. As 
mentioned earlier, information is collected on every 
member of the household. Every person in the 
household is given the household design weight in 
order to eventually derive estimates referring to 


persons. 


3) The LFS is a repeated survey. Once the survey is 
designed, the same design is used month after month 
until a new design is introduced. Historically, the 
survey has been re-designed every 10 years. It is 
expected that growth in the population will occur over 
the life of the design and appropriate adjustments are 
needed at the sampling and estimation stages. 


4) At the time of the design of the survey, information 
from the most recent census is used. In this case, the 
design counts (e.g., the number of households in a city 


block) are from the 1991 census. 


Given the sample allocation, the survey design determines 
an initial set of design weights. These weights could be 
used as long as the design and allocation remain 


unchanged. However, because the penultimate units 
experience growth over time and the systematic sampling 
rate is fixed, this would lead to an ever increasing sample 
size (and ever increasing collection costs). It would also 
lead to large variations in interviewer assignment sizes 
both within the same assignments over time and between 
different assignments. To avoid this, two sampling 
methods are used to control the sample size. The two 
methods, sample stabilization and cluster sub-sampling 
(described below), change a household’s probability of 
inclusion in the sample. It is necessary to adjust the initial 
design weights to compensate for these methods. The 
adjustment factors are called the stabilization weight and 
the cluster subweight. 


Sample stabilization and cluster subsampling involve 
dropping households in order to address problems with 
sample size growth. Stabilization accommodates the slow 
growth over time that is the result of the increasing size of 
the population, which, if left unchecked, would lead to an 
increase in the sample size. Cluster subsampling 
accommodates isolated growth in relatively small areas 
that could present interviewers with work load problems. 


The design weight for a particular household is equal to 
the inverse of the household's probability of inclusion in 
the sample. It is computed as the product of three factors. 
These are referred to as the basic weight, the cluster 
subweight and the stabilization weight. 


The Basic Weight 


When designing the survey, strata were formed by 
grouping together geographic units. Details of the 
stratification can be found in Chapter 2. From each 
stratum, the number of households to be selected is 
determined and fixed. For stratum h we will call this 
number n, . We also know the number of households in 
the stratum at the time of design of the survey. Denote this 
by N,, . The stratum inverse sampling rate (ISR) is given 
as: 


Because the LFS uses multi-stage sampling, it is necessary 
to determine the number of units to be selected at each 
stage of sampling. Consider the case of two stage 
sampling. The expected sample take based on design 
counts from each first stage unit (FSU) is fixed. This is 
called the density factor and for FSU j in stratum h, it will 
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be denoted by n*,, . The number of FSUs to select, nj, is 
given by n,/n*,, . If N,; is the number of households in 
FSU j in stratum h , the sampling rate for the FSU is 
N,,/n*,, - This is denoted by R,,;. R,; corresponds to the 
sampling interval used to systematically select dwellings 
in the final stage of sampling. It is sometimes referred to 
as the cluster ISR. 


In some cases n,, is fixed and n*,; is determined as 
n,/n*,,- In either case, the size of the stratum is set to 
obtain desirable sample sizes. 


We can now determine a _ household’s inclusion 
probability as the product of the selection probabilities at 
each stage. We use R,, as the size measure for PPS 
sampling, for the j" FSU in stratum h. The first stage 
inclusion probability for FSU j is: 
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The conditional inclusion probability of selecting 
household k given that FSU j is selected is, by definition, 
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The inclusion probability of household k in stratum h then 
is 
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The inclusion probability equals the original stratum ISR, 
V/R,, . In general, sample designs with equal basic weights 
within each stratum are called self-weighting designs. The 


LFS is self-weighting within each stratum (with respect to 
the basic weight), and the basic weight is R,. 


The Cluster Subweight 


As described earlier, the LFS follows a multistage design. 
The penultimate units, or clusters, are sampled at a fixed 
rate determined on the basis of the 1991 census counts, to 
yield between 6 and 10 dwellings per cluster. In urban 
areas, new development often takes place and the number 
of dwellings in a cluster can grow substantially over time. 
When this occurs, given the fixed sampling rate, an 
interviewer's assignment size can grow substantially. This 
can affect the quality of the interviewer’s work in addition 
to his/her ability to complete the assignment. When 
growth in a cluster exceeds 200%, the cluster may be 
subsampled using one of three methods. Each of these 
methods involves randomly dropping sampled households 
from the growth cluster, with the result that the household 
probability of inclusion is altered. Instead of continually 
re-computing the basic weight, it is easier to compute a 
weight adjustment and apply it to the original basic 
weight. This adjustment factor is called the cluster 
subweight. The three methods used to drop the households 
and determine the cluster subweight are the following. 


Method I: Subclustering 


When growth exceeds 300% and street patterns are 
defined well enough to delineate clusters, the growth 
cluster is divided into several clusters. A sample of the 
smaller clusters is taken, say n,, of them. The smaller 
clusters are sampled in a manner that will reduce the 
overall take. Let R,, equal the sampling rate of the original 
cluster. The sizes of the new clusters, Nj, and the 
expected sample takes, n,,, give the sampling rates for the 
new clusters, R,,, . We now determine the rate at which we 
have to sample the original cluster to obtain the total 
sample size obtained from the new subclusters. This is 
given by 
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The original basic weights given to the households that 
get selected are multiplied by this factor to reflect their 
actual selection probability. 


Method I: Self- Representing Cluster 


When the characteristics of the growth dwellings are 
distinct from the remainder of the stratum or the size of 
the cluster is at least 20% of the size of a stratum, the 
cluster is first re-classified as a stratum. Call this new 
stratum (hj) . New clusters are formed within this new 
stratum and a sample is drawn. The sample from the 
growth cluster now represents the cluster itself, rather than 
the larger original stratum. If the design count of the new 
stratum is Nj, and the expected sample take is n‘y,) , the 
stratum sampling rate is given by 
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R*,,) is the basic weight to be assigned to households 
selected from this new stratum. Since households selected 
from this new stratum are assigned the weight from the 
original stratum, R,, the appropriate factor is 
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It is also necessary to apply an adjustment to all the 
sampled households in the remainder of the original 
stratum. Consider a stratum from which six clusters are 
selected. The six clusters are used to represent the full 
stratum’s population. After removing the growth cluster 
from the stratum, the weights for households in the five 
remaining clusters must be adjusted so that they represent 
the remainder of the stratum. 


Let N¥, = N, - N\jy be the design count of the remainder 
of the stratum and, if n,, is the original expected take from 
the cluster that has been removed from the stratum, let 
n®, =n, - n,, be the expected sample take from the 
remainder of the stratum. The new inverse sampling rate 
for the stratum is 


i=) 


This leads to the ciuster subweight 


Method III: Cluster Subsampling 


When a cluster is to be subsampled , and neither method 
I nor II applies, this, the simplest and most common case 
of subsampling, is used. First the cluster is sampled based 
on the original design sampling rates. This yields a set of 
sampled households. A second random selection is made 
from the sampled households. The households that remain 
after the second selection are interviewed while the 
remainder are dropped from the sample. If the cluster was 
originally sampled at a rate of R,, , and subsampling leads 
to a sampling rate of R’,, then the cluster subweight is 


For example, if every second selected household is chosen 
to remain in the sample then the new sampling rate for the 
cluster is twice the old sampling rate. The above ratio 
would equal two. For households in this cluster, the basic 
weight will be multiplied by two to compensate for the 
discarded households. Due to outlier problems 
encountered by special surveys that use the LFS frame, the 
maximum value the cluster subweight can be is 3. 


The Stabilization Weight 


The final stage of sampling is conducted using systematic 
sampling at a fixed rate. As the sampling rate is employed 
consistently over time, growth in the population, and 
hence in the number of households, will lead to an ever 
increasing sample size and escalating survey costs. To 
control costs, sample stabilization is carried out. Sample 
stabilization is the random dropping of dwellings from the 
sample in order to maintain the sample at its desired level. 
By randomly dropping dwellings, a household’s inclusion 
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probability is changed. For example, suppose we define a 
stabilization area, a, in which households have a 
probability of inclusion of | in 200 at the time of design. 
If the stabilization area has a desired take of 300 
dwellings, and sampling using the probability assigned 
yielded 350 dwellings, then 50 dwellings must be 
dropped. After dropping the 50 dwellings, the inclusion 
probability is no longer 1 in 200, but rather 3 in 700 (i.e., 
1/200 times 300/350). As with cluster subsampling, it is 
simpler to adjust the basic weights where necessary rather 
than to continually re-compute them. The basic weight is 
retained as 200 but is multiplied by the factor 350/300 to 
yield the desired weight. The adjustment factor is called 
the stabilization weight. 


It is first necessary to define stabilization areas. For the 
present design, a stabilization area is defined as all 
dwellings belonging to the same EIER and the same 
rotation group. For each stabilization area a, a base 
sample size is determined. This is the desired sample 
based on the sample allocation. The base sample size for 
area a is denoted b,. If sampling took place without 
stabilization, a number of dwellings would be selected. 
Call this number n, . If m, exceeds b, it is necessary to 
drop n, - b, dwellings. This is done systematically at 
random. Once this is done we adjust the basic weight. 


The LFS follows the rule that if a cluster has been 
subsampled using method III (see the previous section), 
then the cluster should be excluded from stabilization. No 
dwellings from that cluster can be dropped nor is the 
stabilization weight applied. Denote the total number of 
dwellings in stabilization area a excluded in this manner 
by c,. 


There are two other cases when a household in a 
stabilization area does not receive the stabilization weight. 
On occasion, a group of households that were originally 
believed to be one household are encountered. These 
households, called multiples, are all included in the 
sample. As they did not have an opportunity to be 
excluded via stabilization, they do not receive a 
stabilization weight. Also, over the lifetime of a cluster, 
new dwellings are built and added to the cluster list of 
households. Again, since they were not eligible to be 
dropped, no stabilization weight is applied. 


Once the dwellings have been dropped, the stabilization 
area is partitioned into sub-areas. A stabilization sub-area 
is the collection of strata within the stabilization area 
which have a common inverse sampling rate R, . The 
stabilization weights are calculated separately for each 
sub-area. In the notation we ignore this subtle point. 


The stabilization weight to apply to households in area a 
is 


To conclude this section we repeat that the design weight, 
or inverse inclusion probability, w is given as the product 
of the basic weight R,, the stabilization weight s, and the 
cluster subweight K. 


Treatment of Nonresponse and Derivation of the 
Subweight 


As with all surveys, the LFS experiences nonresponse. 
Nonresponse is classified into one of two types. 


|. Item nonresponse occurs when only some information 
about a household is missing. This could mean some, 
but not all items are missing for one or more 
household members, or all information is missing for 
some but not all household members. 


2. Unit nonresponse occurs when there is no information 
available for any members of the household. 


Item nonresponse is treated entirely by imputation. For a 
particular missing item, a donor record is found among the 
respondents. The donor’s responses to the corresponding 
missing information is used. Typically, a suitable donor 1s 
a person who has similar geographic and demographic 
characteristics and, for those items for which responses 
are available, similar response patterns. The details of the 
imputation method can be found in Lorenz (1995). 


In the case of unit household nonresponse, if a 
nonresponding household had responded in the previous 
month, then the previous month's responses are “carried 
forward”. This method is employed only if there was a 
response in the previous month (1.e., carried forward data 
is not carried forward again). 


Finally, all remaining whole unit nonresponse is treated by 
the method of weight adjustment. The principle of weight 
adjustment is that the responding households can be used 
to represent both responding and _nonresponding 
households. The design weight is multiplied by this 
nonresponse adjustment factor (defined below) and the 
result is called the subweight. 
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To carry out this weight adjustment, the sample is first 
partitioned into weight adjustment classes or nonresponse 
areas. The nonresponse areas are defined in such a way as 
to improve the chances that the respondents will have 
characteristics similar to those of the nonrespondents. In 
the LFS, the nonresponse area is defined as all households 
that belong to the same EJER, the same type of area and 
the same rotation group. Type of area refers to the type of 
frame the sample is drawn from (see Chapter 2). The 
classifications are as follows. 


CMA Apartment design 
CMA Regular design 


Non-CMA Computer 
Program (CADP) design 


Assisted Districting 


Urban EA design 
Urban Cluster Design 
Urban 3 stage design 
Rural EA design 
Rural 3 stage design 
Remote area design. 


Rotation group is included in the definition of a 
nonresponse area because it is known that both the 
magnitude and patterns (refusals, non-contacts, etc.) of 
nonresponse differ depending on how long a household 
has been in the survey. In the context of nonresponse 
adjustment, this is discussed in Kennedy et al. (1994) . 
One feature of the new design is the formation of high 
income strata. Because of their unique characteristics, 
high income strata are treated as nonresponse areas on 
their own. Note that the nonresponse areas do not overlap, 
and together they cover the entire target population. 


Within each nonresponse area, a nonresponse adjustment 
factor is computed. The adjustment factor for a 
nonresponse area is given as the ratio of sampled 
households, weighted using the design weight to represent 
the number of households in the area, to responding 
households weighted to estimate the number of 
households in the area that would respond. If we denote 
by n the number of sampled households in nonresponse 
area b, and by r the number of responding households, 
then the nonresponse factor is 


where 1, is the design weight assigned to the household. 


A value greater than two for the above weight is 
undesirable so when this occurs the nonresponse area 1s 
collapsed with another nonresponse area chosen so that 
when the pooled weight is computed, it will be less than 
two. The nonresponse area to collapse with must come 
from the same province, the same type of frame and the 
same rotation group (collapsing across EIERs if 
necessary). 


This weighting factor is applied to all responding 
households in the area. The subweight is defined as the 
product of the design weight w and the nonresponse factor 
f,,. 


The Final Weight 


In principle the subweight defined above could be used to 
produce estimates of the characteristics desired. However, 
from estimation theory, it is known that if auxiliary 
information about the target population is available, and 
this information is correlated with the characteristics of 
interest, then it can be used to produce more efficient 
estimates. Consider a sample that by chance consists of 
50% women and 50% men. If the true distribution of 
males and females in the population is 51% women and 
49% men, then the sample under-represents females. 
Many labour force characteristics are related to gender. 
For example, a higher proportion of men are employed. 
Adjusting the subweights so that the true proportion of 
each gender group is represented would lead to a better 
estimate. The adjustment factor computed to exploit 
auxiliary information is called the g-factor. The product 
of the subweight and the g-factor is called the final 
weight. 


To obtain the g-factor, the LFS uses a form of the general 
regression estimator (GREG) based on the weighting 
methodology proposed by Lemaitre and Dufour (1987). 
Postcensal estimates of population projected to the current 
time period are used as auxiliary information. Specifically, 
the estimates used are population totals for 30 age/sex 
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groups within each province, as well as population totals 
for Economic Regions and Census Metropolitan Areas. 
These population counts are produced each month by 
Statistics Canada’s Labour and Household Surveys 
Analysis Division. 


The LFS subweight is a household weight. The GREG 
estimator computes a final weight for each household, 
derived in such a manner that the sum of the final weights 
for sampled individuals in a particular age/sex group, or 
in a particular sub-provincial region, agree identically 
with the population estimates used as auxiliary 
information. Also, the estimates of employed, unemployed 
and not in the labour force will sum to the population 
totals used as auxiliary information since everyone in the 
sample belongs to one of these three categories. Because 
the weight is the same for all persons in the same 
household, family level estimates and person level 
estimates are consistent. This was not the case in the 
methodology employed before the Lemaitre-Dufour form 
of the regression estimator was adopted. 


To conclude, the following are some advantages to using 
the final weight step: 


* consistency of estimates with demographic estimates 
of population. 


e an adjustment for coverage error. 


* a common weight for all members of the same 
household. 


e reduction in sampling error of estimates. 


Algebraic Description of Weighting a Record 


The following is an algebraic description of weighting. 
We begin by introducing notation. The LFS sample 
design consists of a nested hierarchy of geography. 


Let p= 1,..., 10 denote the province. 


u=1,..., U denote the EIER u, within province p. 
f= 1, ..., F denote the type of frame within EJER u. 
h=1,..., H denote stratum h within frame f. 

r= 1,..., 6 denote the rotation group within stratum h. 
j=l,..., J denote cluster j of rotation group r. 

k = 1, ..., K denote household k in cluster j . 


... , (, denote individual 1 within household k. 


With this notation a household is identified with the 
subscript pufhrjk . A subscript containing periods or 
missing subscripts indicates a reference to a level of 
accumulation. For example, pu..r refers to all households 
in province p, EIER u, and rotation group r, collecting 
households over the missing subscripts. 


In a few cases, strata cut across EIER boundaries. For the 
most part this has occurred because HRDC redelineated 
its EIERs after the redesign of the LFS. Special estimation 
techniques are used to produce estimates for these regions. 
We note that the above geography is not quite perfectly 
nested. This will present no problem for the standard 
estimation methods discussed in this section. 


At the time of design of the survey, the inverse selection 
probabilities are the same for all households in the same 
stratum. The basic weight can be denoted as 


Wout 


The next two weighting factors, the cluster subweight and 
the stabilization weight adjust the basic weight to account 
for various adjustments to the sample yields as described 
earlier in this chapter. The method of computing the 
cluster subweight depends on the method of subsampling 
employed. 


Method I: Area Subsampling 


In this case, the cluster is redelineated into smaller 
clusters. A sample of clusters is then selected and sampled 
to obtain some fixed total yield. If the sampling rate from 
the original cluster was Rum, and if the sampling rate at 
which the original cluster had to be sampled in order to 
obtain the new total sample yield is R mj then the cluster 
subweight is 
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Method II: Self Representing Cluster 


In this case, the growth cluster is removed from the 
stratum and forms a new stratum. The new stratum, h’ say, 
is delineated into clusters and sampled. If the new strata 
were sampled at the same rate as the old strata, no 
adjustment weight would be required. However this would 
likely yield a very small take. If the sampling rate of the 
original stratum is Ryym, and the sample rate of the new 
stratum is R’ ys, then the cluster subweight to be assigned 
to households in the new stratum only is 
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It is also necessary to adjust the remainder of the clusters 
in the old stratum to compensate for losing a cluster. 
Recall the sampling rate of the original stratum is Ryum- 
Denote by R*®,9, the rate at which the remainder of the 
original stratum would be sampled to get the expected 
design sample take from the remaining clusters. This leads 
to the following factor which is applied to all households 
in the remainder of the stratum: 


Method III: Cluster Subsampling 


In this the simplest case, the selected households are 
subsampled and only the subsampled households 
interviewed. If Ryum,j is the original sampling rate for the 
cluster and Res is the cluster sampling rate required to 
achieve the appropriate level of subsampling, then the 
cluster subweight is 


= Routh 
R 
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As outlined earlier, stabilization weights are computed 
within stabilization areas. In the present design, a 
stabilization area is defined as the set of all strata 
belonging to the same EJER. This area is then divided into 
common rotation groups. Within each stabilization area, 
a base sample size is determined. This is the number of 
households the area should sample based on the sample 
allocation. This number is denoted as b,,,- When 
sampling takes place, a realized number of households is 
encountered, say n,,,- If n,,, > b,,,, then the area is being 
over-sampled and the excess households are dropped at 
random, using systematic sampling. As clusters that were 
subsampled using Method III of cluster subsampling 
above are not eligible for stabilization they are excluded 
when computing the stabilization weight. Denote the total 
of these dwellings in a stabilization area as ¢,, ,. 


When an area is subject to stabilization the following 
factor is applied to households in that area: 


Note that some households in a stabilization area do not 
receive the stabilization weight. These households are 
defined earlier in this chapter. Essentially, they are 
households that were not eligible to be dropped via 
stabilization. 


We can now compute the design weight for each 
household as 
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The design weight is the inverse inclusion probability for 
the given household. When referring to the design weight 
in the following, the cumbersome subscripting will be 
dropped. That is, 


alt Bes 
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The next adjustment is the nonresponse adjustment. 
Nonresponse areas are defined and an adjustment weight 
applied to compensate for complete household 
nonresponse. The LFS defines nonresponse areas as all 
sampled households belonging to the same EJER, the 
same type of frame and the same rotation group. The 
adjustment factor is computed as the weighted ratio of 
sampled households to respondent households. That is, 


em 
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us 
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where summation over s indicates summation over all 
households in the nonresponse area and summation over 
r is over al] responding households in the area. All 
households in the same nonresponse area receive the same 
nonresponse adjustment factor. 


The subweight is given as the product of the design weight 
and the nonresponse adjustment: 
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Note that all members of the same household receive the 
same value of the subweight. 


As mentioned earlier, we could use the subweight to 
estimate the desired characteristics. Given a characteristic 
Y, employment, say, we are interested in the total number 
of persons employed in the population. This can be 
denoted as 


ly yi Dey, 
U 


where summation over U indicates summation over all 
persons in the in-scope population (the subscript i above 
refers to persons ) and y, has a value of one if an 
individual i is employed and a value of zero otherwise. . 


The survey estimate based on the subweights defined 
above would be 


bi = Doyai 
s 


where summation over s indicates summation over 
sampled persons only, and a, is the subweight.It is useful 
to note that, in certain cases, we could re-write the above 
formulae as 


and 


where c¢, is the number of persons in household k , N is 
the number of households in the population and n is the 
number of households in the sample. The y,, are household 


totals 3 y, for the characteristic of interest, such as the 
ick 

number of employed persons in the household. The 

subscript k refers to a household’s total and the subscript 

i to an individual’s value, and abusing the notation, we 

have used / instead of ki. 


The LFS has access to post-censal population estimates 
that are derived independently of the sample. These are 
used as auxiliary information to derive a final set of 
weights. To exploit the auxiliary information methods 
such as poststratification or a regression estimator can be 
used. The regression approach used by the LFS is 
described in Lemaitre and Dufour (1987). In the 
following, the approach used in chapter 6 of Sarndal et al. 
(1992) is used. 


To begin, consider the following notation. 


y,; is the value of the characteristic of interest for 
individual 1. 

y, is the household total of the characteristic of interest 
for household k. 


Q is the number of auxiliary variables used in estimation. 
Each auxiliary variable will be denoted by q = 1,...,Q. 


X, 1S the value of the q'" indicator variable for individual 


i. The indicator variable assumes a value of one if 
individual i belongs to the j auxiliary category and 
zero otherwise. 


X,. is the total of the values of the q" indicator for all 
persons in household k. 


x, isaQx | vector whose q™ entry is the corresponding 
household total x, 


c, is the size of the k household. 
i is the subweight-based estimate described above. 


t,, 18 the known population total for the q'"" auxiliary 


variable. 


t,, is the subweight-based estimate for the q" 
auxiliary variable. 


Thus 
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The regression estimator used can be written as 
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The B, will be defined below. From the above formula, 
we can see that the regression estimator can be viewed as 
the subweighted estimator plus an adjustment term. If the 
sample-based estimate is close to the known total for x,, 
then the adjustment term will be close to zero. If they are 
different, the adjustment term may be large. Use of the 
regression estimator produces gains in efficiency if the 
characteristics are correlated with the auxiliary variables. 


To define the B,, matrix notation is used. 
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where 
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is a QxQ matrix. This matrix is the inverted weighted 
sum-of-cross-products matrix in regression estimation, 
and 


= XY, a, 


k=1 Cc, 


is a Qx1 vector. 


The above estimator can be rewritten as 
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where 
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The g, or g-factors are the factors applied to the 
subweights to obtain the final weights. 


The fact that the final weights do not depend on the 
characteristic y means that the same weight is used for 
tabulating all characteristics of interest. Also note that the 
g-factor is at the household level and each person in a 
household receives the same factor. Alternatively, we 
could compute the g-factors as 
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where a, is the subweight assigned to the i person in the 


sample, and the z,; contain, for every person, the average 
of the values of the indicator variables for each person in 
the same household. That is, 


Every person i in household k receives the same value of 
the indicator variable z, namely the household average. 


Variance Estimation: The Jackknife Algorithm 


The variance estimator implemented in the LFS is the 
jackknife. In the general case, a description of the 
jackknife is found in Wolter (1985). Here we outline the 
jackknife as it is applied in the LFS. The first step in the 
jackknife method is to create replicate samples from the 
LFS data. Within each design stratum a first stage 
sampling unit is selected in turn. This FSU is deleted from 
the sample and the subweights in the remainder of the 
stratum are adjusted to compensate for the deletion. We 
then recompute the final estimates based on the replicated 
sample, i.e., the provincial sample including all but the 
deleted FSU. By repeating this procedure for every FSU 
in the sample, we obtain estimates for every replicate 
sample, as many estimates as there are FSUs. The 
variability among the replicate sample estimates can be 
used to estimate the variance of the sample estimate. For 
convenience, we will refer to the FSU being deleted as a 
replicate. 


To obtain a variance using the jackknife procedure we 
proceed as follows. 
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(i) Remove all the households from a specific replicate. 
Replicates will be denoted by a = 1,... , J,, . That is, 
the h™ stratum contains J, replicates each one 
denoted by a. 


The total number of replicates in the sample is 


where H is the total number of strata in the sample. 


(11) In the given stratum, for all the households in the 
remaining J, - 1 replicates, an adjustment to the 
subweights is made. This is done to compensate for 
dropping the households in the deleted FSU. The 
adjusted weight is 


adj 
a ig ks 


(iii) Using the remaining sample, with the adjusted 
subweights, we recompute the final weights to obtain 
a new estimate of the desired characteristic. The new 
estimate can be denoted as 


A 


che) 


The notation (ha) indicates that the a replicate from 
the h™ stratum was deleted in order to obtain the new 
estimate. Thus the above estimate is based on all but 
the (ha)" replicate. 


This procedure is repeated for every replicate in the 
sample. This leads to J different estimates of the desired 
characteristic. The formula for the variance of the 
estimate of a total is 
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Interest often centres on the ratio of two totals. For 
example, the unemployment rate is the ratio of total 
unemployed to total labour force expressed as a 
percentage. In the general case a ratio 100(y/z)% will use 
the variance formula 
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Variances of the estimates of month-to-month change and 
for averages over a number of months require linking the 
jackknife estimates over time. Consider the difference 
estimate 


and the corresponding jackknife estimates 
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where the superscripts refer to consecutive months. The 
estimate of variance is given by 
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Variance of averages are obtained in a similar way. 
Consider the average over n months, 
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The estimate of variance is given by 
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Changes from the Previous Methodology 


Several changes to LFS weighting methodology were 
introduced with the implementation of the new sample 
design. Most notable were the elimination of the rural- 
urban factor, a change in definition of a nonresponse area, 
a change in the definition of the stabilization area, and the 
elimination of the replication of special area records. The 
methods used to accommodate weighting during the 
phase-in of the new sample are also described below. 


Rural-urban Factor. In the old LFS design, some strata 
in non-self-representing areas consisted of both rural and 
urban parts. This made it possible to have an over- or 
under-representation of the rural or urban population. A 
factor was used to adjust the subweight so that the 
proportion of rural and urban population in any Economic 
Region was the same as it was at the time of the 1981 
census. In the present design, the stratification is explicit 
and hence the sample is representative. Therefore the 
factor is no longer necessary. 


Nonresponse Adjustment. For the majority of unit 
nonresponse, the LFS uses a weight adjustment. Applying 
such an adjustment requires making the assumption that 
nonrespondents can be represented by their responding 
counterparts in the so-called nonresponse areas. The old 
method of defining a nonresponse area was as a stratum 
in self-representing and special areas and as the rural or 
urban part of a primary sampling unit in the non-self- 
representing areas. 


Nonresponse patterns for the survey have been observed 
over time as part of the ongoing quality monitoring 
program. It has long been noted that response patterns 
differ among different rotation groups. A household’s 
tenure in sample tends to affect the magnitude of 
nonresponse. The proportions of the different types of 
nonrespondents (non-contact, temporarily absent and 
refusal) also differ with the tenure in sample. Therefore it 
was decided to include tenure in survey in the definition 
of nonresponse area. Simply adding tenure in the old 
definition would have led to areas that had too small a 
sample take for adjustment. The definition was changed 
to be all households in the same Employment Insurance 
Economic Region, in the same type of frame and with the 
same tenure in sample. 


Stabilization Areas. As with nonresponse areas, a change 
in the definition of a stabilization area was implemented. 
Previously, such areas were defined as all strata within a 
province with the same basic weight. Currently, a 
stabilization area is defined as all households in the same 
Employment Insurance Economic Region and with the 


same tenure in sample. The weight adjustment is then 
computed within the stabilization area, pooling all strata 
with a common sampling rate. This change reflects the 
added importance placed on EIERs in the sample 
redesign. 


Special Area Replication. The old LFS design had three 
frames that were small in terms of population. These were 
the institutions frame, the remote area frame and the 
Quebec remote urban frame. Combined, these frames 
covered about two percent of the population. The cost of 
interviewing in these areas was substantially more than in 
other areas and the sample yields were typically quite 
small. This latter point led to small interviewer 
assignments covering large areas. Because of the 
operational difficulties of small assignments and the fact 
that such small populations were involved, it was decided 
to ignore the subprovincial regions when sampling these 
areas. For example not every Economic Region had a 
representative sample of the special area households, 
though each province did. To correct for this during 
estimation, all the special area records were replicated 
across Economic Regions that had population of the 
corresponding type. The province level weights were then 
proportioned to represent the subprovincial region. For 
example if an Economic Region contained 10 percent of 
the province’s remote population, the sample records for 
remote areas were replicated into that region and their 
basic weight multiplied by 0.1 . 


In the new design populations in institutions are not 
sampled from a special frame. They are no longer treated 
as a special case. The remaining remote area frame is 
sampled in the same manner as in the old design. 
Replication of the records is no longer required as the 
impact on estimates is minimal. 


Weighting During the Sample Phase-In 


The new LFS sample design was introduced by replacing 
the old sample, one rotation group per month, over six 
months. Whenever households selected via the old design 
were due to rotate out of the sample, they were replaced 
with households from the new design. This process began 
in October 1994, with the new sample being fully 
implemented in March 1995. Because of changes to the 
LFS numbering system, new subprovincial geography, 
and modifications to the weighting methodology, some 
special considerations in weighting were required. 
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e No stabilization of the new sample was carried out 
during the phase in period. 


e It was decided to stop using the rural/urban factor in 
October 1994. As mentioned earlier, the new sample 
did not require this factor. Applying it through the 
phase-in period to the old sample only would have 
led to unstable weighting factors. As old sample 
rotated out, the imbalance in rural/urban populations 
for the old sample would have been exaggerated as 
the new sample contribution to these populations 
could not be accounted for. 


e Nonresponse was adjusted using the old method for 
the old sample and the new method for the new 
sample. 


e Special area replication continued to be carried out 
for the old sample only. 


e Once the subweights were computed the two samples 
were combined for the final weighting step. 


New Developments: Composite Estimation 


Until now, the fact that five-sixths of the LFS sample is 
common between consecutive months has not been 
exploited to improve estimates. It is well known that in a 
rotating sample design the common sample can be used to 
produce a better estimate of change compared to simply 
taking the difference between the usual estimates for two 
consecutive months. This improved estimate of change in 
turn can be used to improve the estimate of level. For 
example, the traditional K-composite estimator is a linear 
combination of the usual estimate of level, say a 
regression estimator, and another estimate of level 
obtained by taking last month’s estimate of level and 
updating it using an estimate of change based on the 
common sample, i.e., 


est’ 41) = K x eSty,,) + (1-K) x [est’ + change, onmonl 


where the prime denotes a composite estimate. Although 
traditional composite estimators led to improved 
estimates, they suffered from a number of drawbacks such 
as consistency of estimates. Therefore, the LFS has 
chosen not to implement composite estimation until now. 


The paper by Singh et al.(1997) describes a version of 
composite estimation called modified regression (MR) 
composite estimation. The MR estimator is similar in 
spirit to, but different in detail from, traditional composite 
estimators. In particular, it deals with all characteristics 
that are to be “composited” simultaneously and takes care 


of the consistency issue. The MR method has the 
operational advantage that it fits well into the estimation 
framework currently used by the LFS---the characteristics 
of interest enter into the estimation procedure as control 
totals. It also has two essential properties: each sampled 
household will have a single weight (1.e., the weight does 
not depend on the characteristic of interest) and parts will 
add up to the corresponding total (e.g., the sum of 
employed and unemployed will still equal the size of the 
labour force, which is not the case in the traditional 
approach where each variable is treated separately). 


For the characteristics that are controlled in the MR 
process, there can be substantial improvement in 
efficiency as measured by their variance. For example, our 
studies show that for employment estimates in certain 
industries whose regression estimates are volatile, the gain 
in efficiency can exceed 40 percent. For employment and 
unemployment estimates at the province level, the gains 
are more modest but still worthwhile. For example, for 
Ontario the gains are five and twelve percent, 
respectively. For estimates of month-to-month change, the 
gains can be much more pronounced. For example, the 
variance of the estimate of month-to-month change in 
employment in Ontario is cut in half. For change in 
employment in some industries, the variance is reduced by 
even more. One important consequence of the latter result 
is that certain time series which could not be seasonally 
adjusted effectively in the past will be adjustable when 
MR is implemented, i.e., MR increases the signal-to-noise 
ratio sufficiently to allow the seasonal adjustment 
procedure to detect the seasonal pattern. Based on these 
encouraging results, it is planned to implement MR 
composite estimation in the LFS beginning in the near 
future. 
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CHAPTER 6: Data Quality 


LFS Quality Indicators 


LFS estimates, like those produced from any other sample 
survey, contain sampling errors and nonsampling errors. 
Accordingly, if estimates from this survey are to be 
interpreted correctly, knowledge of their quality is 
required. 


In a sample survey, inferences about the target population 
are drawn from data collected from a single portion 
(sample) of this population. Results are probably different 
from those which would be obtained if a complete census 
of this population were conducted under the same 
conditions. Errors due to the extension of conclusions 
based on one sample to the entire population are known as 
sampling errors. Factors which influence sampling errors 
include the sample size, the variability of the 
characteristics studied, the sample design and the method 
of estimation. 


Nonsampling errors, as the name indicates, have nothing 
to do with the sampling aspect of a survey and can occur 
in a census (in which all units of the population 
participate) as well as in a sample survey. This type of 
error can occur at any stage in a survey (planning, design, 
data collection, coding, capture, editing, estimation, data 
analysis and dissemination) and is primarily due to human 
error. Nonsampling errors can also be associated with 
other types of error such as errors in the population 
estimates used by the survey, errors in sources of 
information and methods to produce population 
projections, errors in seasonal adjustments, etc. 


To ensure and monitor the quality of its data, the LFS has 
an extensive data quality program. A whole range of 
quality indicators are produced on a regular basis and 
carefully analysed. In the presence of unusual values, 
those responsible for the relevant LFS operations are 
immediately advised, to guarantee data quality from one 
survey to the next. Some indicators are also monitored in 
a less regular manner, since their role is to assist in the 
identification of long-term trends or effects, for example, 
the consequences of certain operational or sample design 
changes. This long-term information about reliability of 
data can be used to make changes which will improve the 
general quality of results and help data analysts and users, 
internal and external, in their work. In the following, the 
quality indicators produced for the LFS are presented 
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under two headings: sampling errors and nonsampling 
errors. 


Sampling Errors 


The effect of sampling error on survey estimates is a 
function of a number of factors. The most obvious is 
sample size. All other factors being constant, sampling 
error generally decreases as sample size increases. In 
addition to sample size, sampling error depends on factors 
such as population variability, method of estimation and 
sample design. For a given sample size, sampling error is 
linked to a range of sample design characteristics such as 
stratification method used, sample allocation, choice of 
sampling unit and selection method used at each sampling 
stage for a multi-stage design. In addition, for a given 
sample design, the method of estimation plays an 
important part. Finally, even if the sample size, sample 
design and method of estimation used were the same, 
estimates of different characteristics (for which data had 
been collected from the same sample) would have 
different sampling errors, since the degree of variability 
would vary from one characteristic to another. These 
errors are usually larger for characteristics which are 
relatively rare or distributed unevenly throughout the 
population than for common or homogeneously 
distributed characteristics.. Unemployment estimates, for 
example, generally have a larger sampling error than 
employment estimates, although both are based on the 
same sample. 


Use is usually made of the mean-square error of one or 
more characteristics to measure the efficiency of the 
sample design and of the method of estimation. Mean- 
square error is defined as the average of the squares of 
the deviations of the estimated value of the characteristic 
from its actual value in the population. In sampling 
theory, for finite populations, the average of estimates 
from all possible samples is known as the expected value 
of the estimate. The difference between the expected 
value and actual value is known as the estimation bias. 
The variance of the sample estimate is the average of the 
squares of the deviations of the estimate from its expected 
value. The square root of the variance is known as the 
standard error of the estimate. 


If the method of estimating were not biased, the expected 
value of the estimate and the true population value would 
be identical, as would mean squared error and variance. 
Although certain methods of estimating (such as the one 
used for the LFS) cause a small bias, they result in smaller 
mean squared errors than other unbiased methods. 


One of the important characteristics of a probability 
sample, such as the one used by the LFS, is that the 
variance of an estimator (and therefore the standard error) 
can be estimated from the sample itself. How this is done 
for the LFS is explained in Chapter 5. Here, we use a 
simplified notation for some of the quantities described 
there. 


The coefficient of variation (CV) is another important 
measurement of quality related to sampling error. The 
coefficient of variation, which is obtained by calculating 
the ratio (expressed as a percentage) between the 
estimated standard error of an estimate and the estimated 
value, indicates the degree of reliability of the estimate. 
If Y is defined as an estimate of the characteristic of 
interest and d as the estimated standard error of this 
estimate, the CV is expressed as follows: (d/Y) x 100. 


The estimated standard error (d) may also be used to 
obtain the confidence interval associated with an estimate 
(Y). Confidence intervals are used to express precision. A 
confidence interval is a function of the sample data which 
contains the actual value of a characteristic of the 
observed population with a given degree of confidence. If 
the sampling were repeated many times, it can be asserted 
that 95 times out of 100, the interval Y + 2d would contain 
the actual value. Under the same conditions, it can be 
asserted that 68 times out of 100, the interval Y +d would 
contain the actual value. 


To highlight the links between the various measurements 
of precision, let us take the following example. In March 
1995, the unemployment rate for the Canadian population 
aged 15 and over was 10.8%, and the estimate of the 
corresponding standard error was 0.0016. The estimate of 
the coefficient of variation is therefore (0.0016/0.108) = 
1.48%. The 95% confidence interval, estimated from the 
sample, is between 10.48% and 11.12%, i.e, 
0.108+0.0032. This means that with a 95% degree of 
confidence, it can be asserted that the unemployment rate 
of the target population is between 10.48% and 11.12%. 


Because of the LFS’s very tight monthly publication 
deadlines, for any given month, CVs based on the current 
month are not available for immediate publication. Given 
the stability of CVs observed in the LFS, what is provided 
instead is an estimate of the coefficient of variation based 
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on the average of CVs for the previous six-month period. 
These estimates are updated twice a year (January-June 
and July-December), and appropriate adjustment factors 
are applied to the averages to reflect any change that has 
occurred (for example, a reduction in sample size). 


Data collected by the LFS make it possible to produce 
thousands of estimates of population characteristics. 
There are monthly estimates, estimates of month-to-month 
change, estimates of level averages and estimates of the 
variation in annual averages, at the national, provincial 
and subprovincial levels. Because of space limitations in 
regular and special publications, it is not possible to 
include direct CV estimates for all published survey 
estimates. However, there are look-up tables which 
contain CV approximations for various groups of 
estimates. The following table presents a few 
representative values of coefficients of variation for the 
employment and unemployment characteristics at the 
provincial and national level based on survey data from 
January to July 1997. 


Observed monthly coefficients of variation for 1997 


British Columbia 


There has been an increasing focus on the quality of 
estimates of month-to-month change. To reflect this, the 
monthly LFS press release now includes standard errors 
of change estimates at the provincial and national levels 


design effects based on final weights, since they do not 
take the gain in precision contributed by post-stratification 
into account. 


for employed and unemployed. These are given for the 
1997 period in the table below. 


Standard error of month-to-month change, employed 
and unemployed The LFS uses the sample design effect in conjunction with 
other information to decide in which areas the plan should 
be updated. The following table presents a few values 
representative of sample and overall design effects for the 
unemployed characteristic at provincial and national 


levels. 


Design Effects for Employed and Unemployed -- 1997 


Sample} Overal 
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The design effect is another quality measure obtained 
from the sample. It is defined as the ratio between the 
variance of an estimate resulting from a sample survey 
designed in accordance with a given sample plan and the 
variance of the estimate which would have resulted from 
a simple random sample of the same size. The design 
effect may be used as an index of sample design 
deterioration over time. The LFS computes two types of 
design effect, each dependent on the data used to 
determine it. The sample design effect is determined from 
subweighted estimates, i.e., without weight adjustment to 
take population totals into consideration. The overall 
design effect is calculated using final weights. The sample 
design effect therefore reflects sampling plan efficiency 
only, while the overall design effect provides an overall 
measurement of the strategy adopted by combining all of 
the characteristics of the sample design (stratification, 
multi-stage sampling, post-stratification and estimation). 
The smaller the design effect, the more efficient the 
design with respect to sampling variance. By observing 
the design effect, it is thus possible to measure qualitative 
changes in the plan in question over time. It must be noted 
that sample design effects are usually larger than overall 
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Nonsampling Errors 


Nonsampling errors can occur at any stage in a survey and 
are generally caused by human errors such as inattention, 
misunderstanding or misinterpretation. The impact on 
estimates may manifest itself in bias and/or variability of 
estimates. If the number of observations is large or if large 
areas are involved, the net effect of nonsampling factors 
on variance may be negligible. On the other hand, its 
effect can be large when small areas are involved or when 
the characteristics being studied are rare or about sensitive 
issues. Nonsampling bias tends to occur in one direction. 
It can be attributable to interviewer training or attitude, to 
a fault in questionnaire design or to the method of 
imputation used to deal with nonresponse. All of these 
factors can contribute to an accumulation of errors in one 
direction. 


Nonsampling variance and/or bias may arise from a range 
of sources. In the following, the focus is first made on 
coverage, nonresponse, vacancy, response and processing. 
New types of indicators, which have become available 
since the introduction of the computer-assisted 


interviewing (CAI) mode, are also presented. With these 
new measures, it is now possible to identify certain 
parameters directly related to field interviewers and to 
monitor the performance of the new technology that has 
been adopted. 


Coverage Error 


Coverage errors occur when sampling frame units do not 
appropriately represent the target population when the 
survey is taking place. Units may have been omitted from 
the sampling frame (undercoverage), units not in the target 
population may have been included (overcoverage), or 
units may have been included more than once (duplicates). 
However, undercoverage represents the most common 
form of coverage problem. Overcoverage is not a serious 
problem for the LFS. 


Coverage errors can occur at a number of stages in the 
survey: during frame design, sampling unit definition, 
determination of selection probabilities for sampling 
purposes, or data collection and processing. The LFS 
indicator used to measure coverage error is known as the 
slippage rate. By definition, this rate is the discrepancy 
between LFS population estimates (without external data, 
i.c., based on survey subweights) and the most recent 
census-based population estimates. The discrepancy is 
expressed as a percentage of the census-based estimate. 
The population estimates used in determining slippage 
rates can also contain errors, and these errors then become 
a component of slippage. In the LFS, undercoverage is 
manifested by a positive slippage rate. To reduce the 
resulting bias, sample estimates are adjusted during the 
estimation process to population control totals from 
independent sources. 


The omission of dwellings or individuals in the target 
population, 1.e., the presence of undercoverage in the LFS, 
can introduce nonsampling errors. A dwelling is an 
habitable structure meeting certain criteria. Individuals 
living in a dwelling represent a household. An occupied 
dwelling may be omitted from a cluster list for a variety of 
reasons: omission when the list was drawn up, structure 
under construction during the last check, error in cluster 
delineation, or erroneously classified vacant. Persons may 
also be forgotten in a household because the respondent 
does not divulge their presence or because they have been 
allocated a usual place of residence other than in the 
sampled household. Students are often missed since they 
live elsewhere during their studies, although their usual 
place of residence is in the sample. Then, errors may be 
introduced in estimates if people not included in the 
survey differ from those included. For example, if the 
survey misses highly mobile young persons who have a 
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higher unemployment rate than the young people included 
in the survey, this biases the estimates of the 
unemployment rate downward. Finally, as mentioned 
earlier, population estimates can also contribute to 


slippage. 


Other factors contributing to slippage have been 
identified. For example, population growth occurs 
between redesigns, usually in isolated pockets in a non- 
uniform manner. The sampled areas may under- or over- 
represent the growth that has occurred, or may correctly 
represent the growth. Another example: the adjustment 
used to compensate for nonresponse (see Chapter 5) can 
also have an impact on slippage. If nonrespondent 
households have smaller household size, and if they are 
represented in the sample by households of larger size, the 
slippage rate can be affected. 


Every month, slippage rates are analyzed in detail. They 
are produced regularly for census metropolitan areas, 
economic regions, at provincial and national levels, and 
for twelve age-sex groups in Canada (15-19, 20-24, 25- 
29, 30-39, 40-54, 55+). As part of the latest LFS redesign, 
revised series of LFS estimates beginning in 1976 were 
produced using 1991 census counts, adjustment for net 
census undercoverage, an expanded population universe 
(non-permanent residents are now included) and new 
geography. Before the 1991 Census, population estimates 
did not reflect census undercoverage. The slippage rate 
series have therefore been revised to reflect these changes, 
which is why slippage rates appear to be higher than was 
previously the case. The table below contains average 
slippage rates for the 1997 calendar year. 


Every month, the number of temporary dockets created by 
Regional Offices is monitored, and explanations must be 
provided if there is any substantial decrease. For the 
selection of its sample, the LFS holds a central database 
which contains a list of all eligible dwellings. Six weeks 
before sampling, the list of dwellings to be sampled is sent 
to an interviewer in the field to verify and update its 
content. The sample is chosen from the list at Head 
Office. Although the list has been updated recently, it is 
possible when the time comes to visit these dwellings that 
new dwellings have to be added to the list specifically in 
growth areas. These dwellings are called “temporary” 
because a temporary docket number is allocated to them 
for the current month since they did not appear on the 
database; the situation is restored the following month. 


Average Slippage Rate (%) - Canada by Age Group 


and Provinces - 1997 


Provinces Average 
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As a last indicator of coverage quality, average household 
size is produced occasionally for the LFS target 
population by province and area type: rural and urban. 


The series is evaluated for fluctuations or stability. 


All these indicators point to potential problems with 
sample coverage and make it possible to react 
accordingly. To remedy the situation, or slow its 
progression, interviewer exercises may be contemplated 
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to reinforce their knowledge of household composition 
rules, a bulletin can be distributed to explain slippage or 
the multiple dwelling concept, or a re-listing program can 
be established for a number of clusters deemed to be in 
full expansion. Slippage will always be monitored very 
carefully, because in terms of quality, it translates in the 
LFS into reduced sample size compared to what it was 
when the sample design was developed; it can introduce 
a source of possible bias and variance in the estimates. 
Moreover, despite the application of a method of 
estimation to correct for slippage, a certain persistent 
estimation bias can be anticipated, since the 
characteristics of omitted persons and dwellings can differ 
from those of persons included in the sample. 


Nonresponse 


Each month, during the survey week, interviewers have to 
determine which selected dwellings contain eligible 
persons to the survey. Dwellings are classified non- 
eligible for the survey month for the following reasons: 


- dwelling out-of-scope, i.e., a dwelling occupied by 
persons not in the target population such as members 
of the Canadian Armed Forces; 


- dwelling vacant: unoccupied, seasonal or under 
construction; 


- dwelling non-existent: demolished, converted to 
business location, mobile home moved or dwelling 
abandoned or erroneously listed. 


When a dwelling is identified as being eligible for the 
survey, it is not always possible to conduct the interview 
for the following reasons: 


- nonresponse from household: no one home, 
temporarily absent, interview impossible (inclement 
weather, unusual circumstances in the household, 
etc.) or refusal. 


Moreover, since the introduction of CAI in the fall of 
1993, a new nonresponse code has appeared. This code, 
which previously represented questionnaires not received 
in time for processing because of postal problems, has 
retained that connotation but is attributable to technical 
problems. Such problems include: hard disk crash, tape 
drive system defect, insufficient memory allocation, 
excess heat, power outages, telephone troubles, etc. Most 
of these cases can be solved for the next month, but given 
very short publishing deadlines, it is often impossible to 
solve them during the current month. As a result, these 
cases are considered nonrespondents. This component of 


nonresponse has become almost negligible as experience 
with computer-assisted interviewing has increased. 


The three following approaches are used to compensate 
for nonrespondent units. For households nonrespondent 
for the current month (i.e., unit nonresponse), information 
provided during the previous month will be carried 
forward, if it exists. This procedure cannot be applied for 
two consecutive months, however, and it makes it possible 
to process approximately 30% of nonrespondents. 
Nonrespondent households with no information coming 
from the previous month are compensated for by inflating 
the weight of respondent households which belong to the 
same rotation group, employment insurance economic 
region and type of area (see Chapter 5) with a factor 
equivalent to the inverse of the response rate. The 
importance of bias attributable to nonresponse is usually 
unknown, but it is known to be closely linked to 
characteristic differences between respondent unit groups 
and nonrespondent unit groups. Indeed, this is one of the 
reasons why rotation group was added as an aggregation 
variable when compensating for nonresponse. A number 
of studies have shown that nonresponse behaves 
differently depending on duration of participation in the 
survey. Since the effect of this bias increases with a higher 
nonresponse rate, an attempt is made to keep the response 
rate at as high a level as possible during collection 
operations. 


For partial (i.e., item) nonresponse, an imputation method 
is used. First, deterministic imputation is applied if 
possible, i.e., if responses to other related questions are 
studied and a single value is deemed possible. Should this 
prove unsuccessful, a "hot deck" imputation method is 
used. The new imputation system selects a donor at 
random from current month entries which have passed the 
editing rules. Should the iterative process of seeking a 
donor fail, or should the imputed record fail to meet the 
editing rules, given non-consistency between data 
collected and those imputed after a number of attempts, 
the default record is completely substituted. For the wages 
variable, which was introduced in 1997 with the new 
questionnaire, "warm deck" imputation is used instead. 
For this variable, donor selection is not restricted solely to 
current month data. Values transferred from previous 
months are also examined, since the wages question is 
asked when the household is participating in the survey 
for the first time. It is also impossible to select abnormally 
elevated or low values even though they could be real. 


Vacant and non-existent dwellings do not contribute to 
bias in the sample. However, they do increase sample 
variance, since they reduce the number of households in 
the LFS sample. An error can also be introduced if 
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dwellings are misclassified as vacants. In the LFS, a 
Vacancy Check Program has been set up to obtain 
information related to this type of error. 


Since 1993, the LFS has been subject to Statistics Canada 
standards and guidelines for reporting nonresponse rates. 
Every month, weighted and unweighted nonresponse rates 
are forwarded to Statistic Canada’s central nonresponse 
information database, which is mandated to compile 
longitudinal data for a number of regular surveys. This 
database requires nonresponse rates at the collection and 
estimation stages. Before the redesign, the LFS supplied 
rates for the collection stage only, since these were the 
same as in the estimation stage. With the new 
questionnaire and the new production systems introduced 
in 1997, it is now possible to produce different rates for 
the collection and estimation stages. 


The following table presents average nonresponse rates 
for 1997, plus the minimum and maximum reached during 
that year. The LFS nonresponse maximum is normally 
reached in July and the minimum in October. Since late 
1993, a number of factors have disrupted the LFS 
nonresponse rate series. First, the introduction of 
computer-assisted interviewing generated a new type of 
nonresponse, which was virtually non-existent previously. 
Urbanization of the sample design (introduced gradually 
from October 1994 to February 1995), i.e., a larger 
proportion of the sample is selected in urban areas, also 
had an effect, although negligible, on this series, since 
nonresponse rates are generally higher in urban areas than 
in rural areas. Finally, the new sampling design 
necessitated the hiring of new interviewers who tend to 
obtain higher nonresponse rates during their first six 
months with the LFS. For a historical perspective on 
nonresponse issues in the LFS, see the paper by Sheridan 
et al. (1996). 


Nonresponse Rate (Unweighted), Canada and Provinces - 1997 


Provinces 
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During the tests that preceded the implementation of the 
CAI method, the LFS managers were concerned about the 
possible effects of this change on refusals to participate in 
the survey, due to the presence of a notebook computer at 
the respondent’s household at the first interview, and 
hence more reticence on the respondent’s part. This 
component of nonresponse was carefully examined during 
the tests, and no major increase that could be directly 
linked to the use of computer-assisted interviews was 
demonstrated. 


The LFS refusal rates are usually very low. Canadian 
monthly rates vary between 1% and 2%. Refusal rates at 
the provincial level are ordinarily in the same range, but 
can go as low as 0.5% or rise as high as 3%. Indeed, for 
the first time in LFS history, a computerized collection 
mode made it possible to use automated forms that 
mentioned reasons for refusing to participate in the survey 
such as: do not have time, do not believe in statistics, too 
personal, against the government. Studies can now be 
undertaken more readily to identify these causes, and an 
attempt can be made to minimize the number of refusals. 
The case management system specific to the computerized 
collection mode can also indicate the number of refusals 
converted into respondents by senior interviewers each 
month. This information was previously non-existent. 


Vacancy 


Dwellings correctly identified as vacant or non-existent do 
not introduce any bias to LFS estimates. On the other 
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Average (%) | Maximum(%) | Minimum (%) 


hand, estimation variance is higher, since the sample 
includes fewer households. LFS interviewers return to 
vacant dwellings every month to interview persons 
targeted by the survey who may have moved in since the 
previous survey. Non-existent dwellings are simply 
removed from the sample frame. Particular attention is 
paid to the identification of vacant dwellings, since these 
directly influence two other indicators. Should a dwelling 
be coded vacant rather than temporarily absent for 
example, the nonresponse rate produced for the LFS is 
somewhat under-estimated. And the slippage rate is 
overestimated, since this miscoded dwelling should have 
been considered when this rate was being determined. 
Interviewers must therefore take great pains to determine 
whether a dwelling is vacant, and accordingly outside the 
survey field, or simply occupied by a temporarily absent 
household and therefore in the scope of the survey. 


The next table presents average vacancy rates and 
minimum and maximum values for 1997 at the provincial 
and national levels. Once again, to meet Statistics Canada 
standards and guidelines for reporting nonresponse, 
weighted and unweighted rates are produced every month 
and forwarded to the Statistics Canada _ central 
nonresponse information database. In general, the vacancy 
rate is relatively stable, posting an upward trend with 
further distance from the last redesign, since the sampling 
frame is less up-to-date. After each redesign, the vacancy 
rate shows a downward trend. This downward trend was 
even more marked after the last redesign, given the more 
urbanized nature of the new sample design. For this 


estimates may vary slightly from one rotation group to 

another. This leads to what is known as rotation group 

bias. As regards the LFS, this bias, as measured by the 

rotation group index, generally peaks for the one-sixth of 
the sample in its first interview. The rotation group 
index is the ratio between an estimate calculated for the 
sample portion participating in the survey for a certain 
number of times (first month, second, etc.) and the 
estimate calculated for the entire sample. 


quality indicator, certain provinces stand out because of 
much lower or higher posted rates. 


Vacancy Rate (Unweighted), Canada and Provinces - 
1997 
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Brisebois and Mantel (1996) computed a modified 
rotation group index which adjusts for differences in the 
sampling error effects for the six month-in-sample 
groups. Their study, which is based on sample weights 
before adjustment for demographic controls, found 
several statistically significant differences among 
rotation groups, but their practical effect was minor. In 
practice, published estimates are based on weights that 
have been adjusted for age-sex and geographic 
population controls. In addition, the weights for each 
rotation’s groups sample are adjusted to the total 
provincial population. Finally, nonresponse adjustment 
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Response Error 


This error may be attributable to questionnaire design, 
question formulation, respondent understanding, the 
manner in which the interview is conducted or the general 
conditions under which the survey is conducted. 
Response errors can occur when information is provided, 
received or entered in the portable computer. However, 
the computerized collection mode makes it possible to 
reduce some of these errors, since now certain editing 
rules are incorporated in the collection instrument and 
conflicts must be resolved during the actual interview. 
Nevertheless, the respondent may misinterpret the 
question, not know the answer, have forgotten or prefer to 
communicate a distortion of the facts for personal reasons. 
Moreover, interviewers may tend to explain answers or 
interpret them differently. Response errors, like other 
categories of error, can have variance and bias. 


Proxy responses obtained when collecting information 
from a household member about another member can also 
lead to response errors. 


In repeated surveys, where the sample is made up of a 
number of panels or rotation groups, the expected value of 
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is now done separately by rotation group, unlike in the 
period studied by Brisebois and Mantel. As a result, the 
effect of rotation group differences based on these final 
weights is likely to be even smaller. 


Processing Errors 


Processing errors can occur at various stages in the 
survey: data capture, editing, coding, weighting or 
tabulation. 


Using a computerized collection mode makes it possible 
to avoid routing errors on the questionnaire, since the 
application determines the next question to be asked, 
taking previously entered responses into account. In 
addition, certain editing rules are incorporated in the 
collection system, making it possible to detect and correct 
some discrepancies at the time of the interview. However, 
it was not possible to incorporate all the editing rules in 
the computerized collection instrument, since a 
compromise had to be reached between interview length, 
computer speed and computer efficiency. Head Office 
completes this step by applying a set of batch editing 
rules. 


The field edit module provides two types of quality 
indicators: edit failure rate and edit discrepancy rate. The 
edit failure rate represents the percentage of forms with at 
least one discrepancy. A discrepancy is defined as an 
entry that was blanked out, modified or inserted into a 
blank field after performing certain edits to check for 
validity. The edit discrepancy rate represents the 


percentage of discrepancies on a form compared to the 
total number of entries on the form. Edit failure rates of 
approximately 1% and 5% are observed for demographic 
and other items, respectively. For edit discrepany rates, 
the corresponding figures are 0.1% and 1%. 


Automatic and manual coding of occupation and industry 
are carried out at Head Office. During the first interview, 
or in the presence of any change for these variables, the 
interviewer gathers information describing precisely the 
kind of business, industry or service where the person 
works, and information indicating clearly and accurately 
the kinds of work or duties. The first type of information 
is used to determine type of industry, while the second 
type serves to identify the occupation. One of the first 
processing operations at the Head Office consists of 
coding automatically the descriptive information collected 
for the industry and occupation variables according to the 
standard classification for these variables, namely SIC and 
SOC. Records that cannot be coded by the automatic 
system are coded manually by a team of LFS coders. 
Approximately 14,000 records are coded manually every 
month. To control manual coding quality, a statistical 
quality control plan is applied every month. The three 
measures used to determine the efficiency of this control 
process are verification rate, which is the percentage of all 
control-subject entries verified, average incoming quality 
(AIQ), which is an estimate of the percentage of records 
containing errors before quality control, and average 
outgoing quality (AOQ), which is an estimate of the 
percentage of entries still containing coding errors after 
quality control. In 1995, the average values of these three 
measures were 19.6% for the verification rate, 7.9% for 
AIQ and 4.8% for the AOQ. 


To avoid errors likely to occur in estimation and 
tabulation, this stage is followed by a detailed examination 
of the result of these operations, analysis of the various 
diagnostics produced automatically by the system, and 
comparison with other data sources. 


New Error Measurements 


Adoption of the computerized collection mode in the fall 
of 1993 generated a whole range of data that had 
previously been difficult to access. The CAI mode 
features a case management system whose primary 
functions involve routing cases, reporting and assisting 
interviewers in administering the survey. All case 
activities are now recorded. Accordingly, during every 
survey month, files containing a host of information on 
what is happening in the field are directly produced by the 
case management system. For example, it is possible to 
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obtain average time per personal or telephone interview, 
number of attempts to reach a respondent, best times 
(hour and day) for an interview, etc. It is also possible to 
check whether collection procedures are being followed 
to the letter by interviewers. For example, nonresponse 
codes meaning “no one home” must only be used at the 
end of the survey week. This new system makes it 
possible to determine more about the work being done in 
the field, and to react to questionable cases (such as, for 
example, interviews completed in under a minute), and 
accordingly, to minimize certain errors that can slip by. 
This new information can also be used to improve the 
interviewer training program and reinforce certain 
behaviours such as task planning or work scheduling. This 
database also contains the number of cases converted 
from refusal to respondent, which was not available 
previously. The case management system records all the 
activities undertaken in a case. A working group is now 
developing an operational report designed to inform 
Regional Offices (ROs) about any recorded situation 
deemed unusual during the survey month (e.g., interviews 
before the survey week, insufficient time between two 
personal interviews, night call, etc.). To be effective, this 
report must be produced as soon as possible after the 
survey week to deal with the memory effect when looking 
for causes in contacting interviewers. 


New case management data also contain more information 
about editing rules followed during interviews. With the 
new system it is, for example, possible to determine how 
many editing rules have been overridden after 
confirmation with the respondent, how many times an 
editing rule has been applied and how many times an 
observation has failed the rules. 


With the introduction of the new questionnaire and the 
new computer system, it has also become possible to 
produce an overall imputation rate, per questionnaire and 
per question, which was impossible before. 


All these new measures will clarify what is happening in 
the field and help in the interpretation of results. 


LFS Data Quality Reports 


A number of reports are available to LFS data users and 
everyone involved in the survey, at Head Office in Ottawa 
and in the regions. Moreover, these documents are 
consulted regularly by members of the LFS Data Quality 
Committee to monitor survey quality. In general, they 
contain a range of quality indicators at different 
geographic levels and for varying periods. 


Monthly LFS Operations Report. Every month, the 
LFS Data Quality Unit produces a report on current 


month survey data quality. This report is discussed on the 
day before the survey data release. Its primary aim is to 
permit control of operations quality in the field, which is 
why most quality indicators are presented by Regional 
Office. Some series are also presented for a 26-month 
period to depict seasonal and monthly changes in 
comparison with a previous year. The report contains the 
following quality indicators: nonresponse rate (by RO, 
component, number of months in survey, area type), 
vacancy rate (by RO, area type), slippage rate (by 
province and age-sex group), sample size, number of 
technical problems and number of temporary dockets. 


Variance Tables. The variance tables, produced every 
month, contain estimates, CVs, variances and design 
effects of the main LFS characteristics at a range of 
geographic levels. In addition to these indicators, the 
booklet also contains average household size and slippage 
rates at more detailed levels than those presented in the 
LFS Operations Report. At present, some thought is being 
given to opting for a modernized, exclusively electronic 
format. 


Quality Report. The LFS Quality Report is produced 
twice a year, covering the January-June and July- 
December periods. The objective of this report is to 
present an in-depth examination of quality measures 
associated with the LFS for the six-month study period. 
Quality measures are also analyzed over a 30-month 
period to identify the trends or effects of certain 
operational or sample design changes. This document 
focuses on a number of quality indicators presented in the 
Operations Report, but this time at the provincial rather 
than regional level. In contrast to the Operations Report, 
the Quality Report does include text commenting on the 
various tables and charts. In addition to these regular 
indicators, there is an occasional special chapter dealing 
with a subject of particular interest. 


Coefficient of Variation Updates. In line with Statistics 
Canada policy on dissemination quality standards, the 
LFS produces monthly variances corresponding to the 
estimates it produces. Six-month CV averages are updated 
twice a year. 


Special Reports. In addition to reports produced 
regularly to ensure and control the quality of LFS data, 
certain special reports are written occasionally. An 
example is the report dealing with the impact on LFS data 
quality of the introduction of CAI (Simard and Dufour, 
1995). Since this introduction was gradual, it was possible 
to compare the portions of the sample interviewed with 
computer-assisted and paper and pencil interviewing 
modes with respect to a number of quality indicators. The 
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amount of significant data on the subject gave rise to a 
substantial special report. This was also the case for the 
report dealing with the impact on LFS data quality of the 
phase-in of the new sample (Dufour, Simard, Allard and 
Ray; 1996). 


Quality Assurance Programs 


Over the years, the LFS has equipped itself with a number 
of programs to ensure the quality of the data it 
disseminates. Some of these programs have had to be 
dropped because of budget restrictions. Eight programs 
are dealt with in the following. 


Recruiting. Before applicants are hired as interviewers, 
their aptitudes and ability to complete survey documents 
are evaluated. Even before training begins, they are sent 
copies of documents which introduce Statistics Canada 
interviewing work, outline the responsibilities, techniques 
and skills required of an interviewer, describe Statistics 
Canada organization and present the portable computer as 
a collection instrument. 


Training. The initial training period for interviewers 
extends over two months. It begins with a three-day 
classroom course where newly-hired interviewers are 
shown how to use their notebook computer and computer 
equipment, and how to complete survey forms and 
administrative documents. They also carry out practical 
exercises, simulate interviews and learn interviewing 
techniques. 


Interviewers then receive two days of workplace training 
during the first survey week in which they work, and a day 
or two during the second week as required. During this 
time, a senior interviewer accompanies and observes 
them, indicates how to conduct interviews, and sets an 
example by conducting interviews him/herself. 
Interviewers also participate in special group training and 
retraining sessions, at least once a year. 


Interviewers’ work is evaluated as part of other programs, 
which will be described below. Depending on individual 
performance, it is determined whether they need further 
self-instruction courses or revision exercises to clarify 
certain points or remedy weaknesses. 


Observation. The observation program is designed to 
minimize potential interviewer errors by giving senior 
interviewers an opportunity to observe those who report 
to them, to evaluate their performance and to identify 
problems. Each interviewer is observed at least once 
every 24 months. Regional Offices decide who will be 


observed and when, in such a way that it is not possible to 
guess the order in which the observation program will take 
place. Apart from this program, a senior interviewer can 
observe an interviewer if a specific problem is suspected. 
The senior interviewer accompanies the interviewer for an 
entire day and observes how personal and telephone 
interviews are conducted. On the second day, the senior 
interviewer checks the cluster list. Later, the senior 
interviewer sends the results of the observation to the RO 
and writes periodic reports for Head Office. The senior 
interviewer forwards the interviewer’s performance 
evaluation to him/her as soon as possible after the 
observation. 


Performance Feedback. Every month, interviewer 
performance reports are sent to ROs. These reports deal 
with costs, rejection rates on editing and response rates. 
Senior interviewers are regularly in contact with their 
interviewers and bring to their attention the results of a 
range of performance indicators. 


Vacancy Check Program. The purpose of the vacancy 
check program is to monitor interviewers’ field work. A 
sample of vacant coded dwellings is selected at least once 
every 24 months for each interviewer. A_ senior 
interviewer returns to these dwellings to check whether 
the dwellings were actually vacant, and the dwellings are 
recoded (vacant or otherwise). Following on the results of 
this program, the interviewer will receive additional 
training when necessary. This information is also used to 
measure how many households coded outside the survey 
field contributed positively to slippage, although it is very 
difficult to extrapolate for the complete sample, since the 
selection of checked dwellings is left to RO discretion. 


Validation Program. The validation program was 
designed to monitor interviewer performance and to 
provide interviewers with correctional feedback in the 
form of additional training depending on identified 
weaknesses. Interviewers are validated randomly in such 
a way that during the course of a year, each of them is 
selected twice. Approximately 2% of households are part 
of this program every month (except April and December 
when the program does not take place). The week after the 
survey week, senior interviewers recontact the persons 
who provided information during the survey week for the 
sample involved in this program. They ask polite 
questions such as confirmation of address, memory of 
participating in the survey, interview time, interviewer 
attitude, etc. The senior interviewer also takes the 
opportunity to thank respondents for their participation 
and time. This program was suspended during the 
introduction of the CAI collection mode, but will return in 
computerized form. 
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Re-interviewProgram. The Labour Force Survey re- 
interview program is designed to measure the response 
variability associated with data collected by the survey 
and to determine the causes of response errors and 
conditions under which these errors are likely to occur. In 
addition to respondent and interviewer errors, the term 
“response error” includes errors due to questionnaire 
flaws, such as awkwardly phrased questions or inaccurate 
response codes. This program was suspended in the early 
1990s and at the moment, its future is uncertain. 


Periodic analysis of data obtained from this program 
made it possible to discover the causes of errors and to 
improve all methods used on an ongoing basis, as there 
were impacts on a range of survey design elements such 
as questionnaire composition, interview techniques, 
interviewer training and still more. Following is an 
explanation of the program. 


Every month, except December, a number of dwellings in 
the LFS sample were chosen for the re-interview program. 
This subsample was selected so that part of each 
interviewer’s assignment was re-interviewed at least twice 
a year. RO supervisors, or senior interviewers, carried out 
the re-interviews at the selected dwellings by telephone 
during the week following the survey week, taking into 
account the fact that it had been two weeks since the 
reference week. 


The re-interview sample was divided into two parts. For 
one of them, the senior interviewer compared responses 
obtained on the second interview with those provided to 
the interviewer the first time, and if there were 
differences, determined the correct response with the 
respondent's help. For the other part of the sample, the 
senior interviewer merely conducted an independent 
interview. Data collected from the part of the sample 
which was the subject of comparison were used to obtain 
a measurement of response bias, the hypothesis being that 
the interviewer and senior interviewer would not have 
made the same mistakes and that comparison made it 
possible to discover the correct answers. Data collected 
from the part of the sample which was not the subject of 
comparison were used to obtain a measurement of 
response variance. Comparison of responses also 
permitted the senior interviewer to check interviewers' 
work. After the re-interview, the senior interviewer 
discussed discrepancies in results with the interviewers, 
advised them and pointed out the questions with which the 
greatest number of errors were associated. 


Cluster Yield Monitoring. LFS cluster yield is 
monitored on a monthly basis to detect any divergence 
between the number of dwellings enumerated in the field 


and the number of dwellings used in developing the 
sample design. The sample design uses a number derived 
from counts available from previous census data. 
Accordingly, any major difference, i.e. any 50% (positive 
or negative) discrepancy between enumeration and 
derivation, is the subject of an investigation. First, any 
cluster with an unexpected yield is brought to the attention 
of the unit responsible for sample control in Ottawa, 
which checks cluster boundaries and the expected number 
of dwellings. If the discrepancy cannot be explained at 
Head Office, the cluster is forwarded to the Regional 
Office concerned for detailed analysis. All causes 
explaining discrepancies are stored for future reference. 


This monitoring plays an important role, since if the 
sample size necessitates changes, it is essential to 
determine which areas are being under- or over-sampled. 
Moreover, recorded discrepancies can reveal survey 
problems that could mar the quality of LFS data. 


LFS Committees 


The LFS has a number of coordinating groups to ensure 
continued good functioning of the survey. Certain 
committees play a permanent role at the LFS, while others 
are active only during the redesign. During the last 
redesign, there were two main committees : the Redesign 
Steering Committee and the Redesign Operations 
Committee. The first of these two Committees, chaired by 
the director general of the Labour and Household Surveys 
Branch, was mandated to oversee the overall redesign 
project, i.e, to monitor activities related to the 
introduction of CAI, the sample redesign, the new 
questionnaire, new processing systems and new outputs. 
The Redesign Operations Committee, chaired by the 
director of the Household Surveys Division, was in charge 
of managing and controlling work on individual projects. 


In the following sections, three permanent committees are 
discussed. Their mission is to deal on a regular basis with 
permanent operations and evaluation of the survey. 


Operations Committee. This committee’s mandate is to 
review events occurring during survey months and 
circumstances surrounding the carrying out of the survey, 
to see that operations run smoothly, to examine proposed 
changes and to recommend their adoption, with a view to 
ensuring that the survey continues to reach its objectives. 
The committee, which meets every two weeks, is chaired 
by a member of the Household Surveys Division. 


Population Estimation Steering Committee. This 
committee’s mandate is to monitor postcensal population 
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estimates required by the LFS. This committee also 
evaluates data sources and methods used to obtain the 
estimates at different geographic levels, and initiates a 
number of research projects on the topic. A member of the 
Labour and Household Surveys Analysis Division chairs 
this committee. 


Data Quality Committee. When this committee was 
officially struck in the spring of 1972, its function was to 
disseminate the quality of data from the LFS and its 
supplements. Since that time, the committee's mandate has 
expanded somewhat. Its current mandate is to examine 
and evaluate the quality of LFS data on a monthly basis, 
to propose and review research and development projects 
designed to fine-tune methods that could influence data 
quality, and finally to oversee research and development 
in this field. The committee is chaired by a member of the 
Household Survey Methods Division. 


To ensure that LFS data are of the best possible quality, 
the Data Quality Committee regularly reviews the various 
quality indicators mentioned earlier. It meets every month 
to examine and evaluate the quality of monthly data and 
to make suggestions and recommendations on any aspect 
affecting the quality of data that ought to be improved. A 
number of documents are made available to committee 
members to help them in their task. By closely following 
the evolution of these indicators, the committee can 
intervene immediately with those responsible for the LFS 
operations concerned to control monthly data quality. 
The committee also discusses new facts likely to influence 
the quality of data that have just been collected or are to 
be collected in the future, particularly changes to 
collection methods or to the questionnaire, unusual 
problems in the field, ongoing testing of procedures and 
methods, etc. 
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Appendix A 


Table A0-1: 1998 LFS Sample by Province and other Breakdowns’ 


| Strata Households i Source 
Province; Urban|_ Rural| Apartment; Urban) Rural] Apartment Total| Core El 
Newfoundland 35 13 Oia 1154 836 0 1987 1987 0 
Prince Edward Island 17 12 0 603 818 0 1421 1421 fo} 
Nova Scotia] 46 27 Pécs Aes) i 7G! 90 3557 2611 946 
New Brunswick 45 20 0} 1800) 1261 0 3061 2604) 457 
Quebec| _209 38 12| 7693) 2383 283) 10358 5458/4900 
Ontario 298 40 34; 11717; 2158 1634) 15508 7179} 8329 
Manitoba 59 14 6} 2481) 1152) 273| 3906 3255 651 
Saskatchewan 56 21} 1] 2471) 1537) 63 4072 3408) 664 
Alberta Tas. 13 6| 2903 914 174 3991 3991 0 
British Columbia 114 12) 7| 3882 854). 234, 4969) 4113 856 
Canada 954 210 68} 36454; 13627 2751} 52830} 36027) 16803 
Table A0-2: 1998 LFS Sample by EIER and other Breakdowns’ 
Strata | Households __ |__ Source 
EIER| Urban| Rural) Apartment) Urban| Rural)/Apartment| Total} Core El 
020] 15 0 0 465 O| 0; 465) 465 0 
021 20 13 ) 686} 836 O| 1522) 1522 0 
123 17 12 0 603} 818 O| 1421) 1421 0 
224 8 2| 0 294, 145 0} 439) 397 42) 
225 i, 6 0 287| 342 OlF 1629) G55 74 
226 22 0 2 716 0 90; 806| 806) 0 
227 4 f 0 235} 466 0; 701 490) 211 
228 5| A2 0 zen, 7G O| 982; 363) 619 
329) 32 4 Oi 1270). 225 0} 1495) 1090) 405 
| 330) 13] 16 0 530; 1036) O| 1566) 1514 52 
433 8 9 0 323} 541 O| 864 455) 409 
434 We 0 2 730 0 45 |i 314-510-265 
4551t 55 0. OMe 566) ta 880): 0| 566 O| 566 
436 3 3 0 238| 368 O| 606) 224) 382 
437 13 0 0 722| 0 Ovrnii2e 153] 569 
438 13 5 0 590} 325 O| 915) 401 514 
439 80 0} =A ae he Feat 0 2027 27st eclol 42 
440) 20 15 | 0 667| 670 O| 1336) 732} 604 
441 6 3 0 305} 300 OS 6099295) 5310 
442 9 3} 0 507) 179 O| 686; 316; 370 
443 12 0 1 534 0 S657 0 169} 401 
444 13 0| 0 540 0 0; 540 72| 468 


* Breakdowns by Core and EI are approximate. 


,.. continues... 


1998 LFS Sample by EIER and other Breakdowns’ (continued) 


Strata _ eu Households Source 
EIER| Urban) Rural| Apartment, Urban) Rural) Apartment, Total Core El 
546 18 0 S| 684) 0 191 S75 Ola 304 
547 16 9 0 5534) 527 0 1080 387 693 
548 29 14 0 901; 561 O}.), 1462) a 657). 805 
549 18 0 2 780 0 94 874 132,742 
550 66 0 16, 1971 0 827, 2798 2798 0 
55] 14 0 4 604 0 133 737, 448 289 
552 17 0 Tes 760 0 56 816) 86°258)h" 558 
553 15 0 2} 641 0) BRS Gen G5) bers 0 516 
554 v4 4 0} 290 185 0! 475} 136 339 
555) 11 0 2 558 0, 81/638) 167, 471 
556 ie 0) 2 720 0 95 B1S 250 S77, 
Sor 5 2) 0 353,227 0 580, 262 318 
558 20 8 0} 86843, ~—529 0} 1372} 439 933 
| 559 WA 0 0 760 0) 0 760 73 687 
560) 15 0 Ogee tie 0 0 712 52) 660 
561 13 3 0 587 129 0 746). -2(915 437 
3 664 45 0 6 1622 0 273| 1895) 1895 0 
665 10 11 0 522; 742 QO} 1264 1139 125 
666 4 3 0 337, 410 0 (AT 22liz 526 
767 18 0 0} 767 0 0 767 ~~ S626 205) 
768 16 0 1 709 0 63 773,623) 150 
769 14 17] 0 581; 1023 O| 1604 1604 0 
770 8 4 0 414 514 0 928, 619 309 
871) 25 0} 3; 861 0) 85 946, 946 0 
872 29 0 3 970) 0 89} 1059} 1059 0 
873) 21 13) QO} 1072) 914 0 1986, 1986) 0 
975 42 4 0) 530,218 0 748 748 0 
976 60 0) 6| 1573 0 196, 1769, 1769) 0 
977 13 0 die O te 0 38| .. 649... .351)-_-298 
978 21 4 0 628 356 0} 6984 = 627, ~— 357 
979 8 4 0 539 280) Oo 819 618 201 
Canada 954 210 68| 36454) 13627 2751; 52830 36027) 16803 


* Breakdowns by Core and E] are approximate. 


Table A1. Stratification Results for High Income Strata in the LFS Redesign 
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Table A2. Provincial sample allocations and CVs under old design, redesign and current (post-reduction) 


Redesigned Sample Current Sample 
ane 
1,884 1,884 
1,421 


Old Sample 


fe 
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Newfoundland 


Island 


Manitoba 


5 
| 
15,85 
| 
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i 2,98 


| 
52 | 408 | 


#2310 58,060 42,310 58,860 35,810 


52,350 


Table A3.1. Sample allocation by Economic Region, July 1995. 


Province Sample Province] «ER Sample Province| __—ER | Sample 


Newfoundland 68 Quebec Manitoba | ine SOOl Ne 273) 


20/200} (cont)|_ 455] cont. wer Bes 
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pay 
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Nova Scotia 732 
zo| ea 750/760|__709| 

230 Ontario 510 Alberta 810 

820| _300! 

= arc 830| 1027 

New Brunswick 530 840 
540| 2193] 850| _300! 

20] 67 550| __ 1988] geo] 1114 

| 560| _ 1102 870| _ 300 

so] era 570| _ 905) 300 

Quebec 580| 514, British 
590|_ 1740] ~~ Columbia 

pool aaa 595| 885 : 

425| __462| Manitoba 610| 320 

430| 910 620| 200 

435| __1387/ 630| 374 

440} __1198 640| 200! 

=e Total = 5235 


Table A3.2. Sample size allocation by Employment Insurance Economic Region, July 1995. 
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Appendix B: Abbreviations 


CA 
CADP 
CD 
CMA 
CSD 
CT 
cy 
EA 
EIER 
ER 
FSU 
HRDC 
ISR 
LFS 
PPS 
PSU 
RHC 
RO 
RPPSS 
SNF 
UIR 
VR 


census agglomeration 

computer assisted districting program 
census division 

census metropolitan area 

census subdivision 

census tract 

coefficient of variation 

(census) enumeration area 

Employment Insurance Economic Region 
Economic Region 

first stage unit 

Human Resources Development Canada 
inverse sampling ratio 

Labour Force Survey 

probability proportional to size 

primary sampling unit 
Rao-Hartley-Cochran (random group method) 
Regional Office 

randomized PPS systematic 

Street Network File 

Unemployment Insurance Region 


visitation record 
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Appendix D: Urban Area Designs 


Notation: E denotes EA design, V denotes VR design. The 
terms EA, VR and CADP are explained in the document. 


CMAs: All CMAs have CADP clusters. Most also have some 
EA-based clusters; -E denotes absence of EA-based 
clusters. A few CMAs have some VR-based clusters. CMAs 
with apartment strata and high income strata are noted in the 
body of this document. 


St. John’s, Nfld; Halifax, NS; St. John, NB; 

Quebec: Chicoutimi, Montreal, Quebec, Sherbrooke, Trois 
Riviéres (V) 

Ontario: Hamilton (-E), Kitchener (-E), London (-E), Oshawa 
(-E), Ottawa-Hull, St. Catharines (-E), Sudbury (V), Thunder 
Bay, Toronto, Windsor 

Winnipeg, MB; Regina, SK; Saskatoon, SK; Calgary, AB; 
Edmonton, AB; Vancouver, BC (-E); Victoria, BC (-E) 


CAs containing CADP clusters 
New Brunswick: Fredericton, Moncton 


Ontario: Belleville (some E), Brantford, Guelph, Kingston 
(some E), North Bay, Peterborough (some E), 
Sarnia-Clearwater, Sault Ste. Marie, Stratford, Woodstock 


Alberta: Lethbridge, Red Deer 


British Columbia: Kamloops, Kelowna (some E), Matsqui 
(some E), Prince George 


CAs not containing CADP clusters 


Newfoundland: Corner Brook (E), Grand Falls-Windsor (E), 
Gander (V), Labrador City (V) 


Prince Edward Island: Charlottetown (V), Summerside (V) 


Nova Scotia: New Glasgow (E), Sydney-Sydney Mines (E), 
Truro (E), Kentville (V) 


New Brunswick: Bathurst (E), Campbellton (E), 
Edmunston (E) 


Quebec -- EA design: Alma, Baie Comeau, Cowansville, 
Drummondville, Granby, Joliette, La Tuque, Magog, Matane, 
Rimouski, Riviére du Loup, Rouyn-Noranda, Saint Georges, 
Saint Hyacinthe, Saint Jean sur Richelieu, Salaberry de 
Valleyfield, Sept Iles, Shawinigan, Sorel, Thetford Mines, 
Val d'Or, Victoriaville 

Quebec -- VR design: Lachute 


Ontario -- EA design: Barrie, Brockville, Chatham, 
Cobourg-Port Hope, Collingwood, Cornwall, Elliot Lake, 
Haileybury, Kenora, Leamington, Lindsay, Midland, Orillia, 
Owen Sound, Simcoe, Timmins 

Ontario -- VR design: Kirkland Lake, Pembroke, Tillsonburg 


Manitoba: Brandon (E), Portage La Prairie (V), 
Thompson (V) 


Saskatchewan -- EA design: Moose Jaw, North Battleford, 
Prince Albert, Swift Current, Yorkton 
Saskatchewan -- VR design: Battleford, Estevan, 


Lloydminster, Weyburn 


Alberta: Fort McMurray (E), Grand Centre (E), Grande 
Prairie (E), Lloydminster (E), Medicine Hat (E) 


British Columbia -- EA design: Campbell River, Chilliwack, 
Courtenay, Cranbrook, Duncan-Chemainus, Nanaimo, 
Penticton, Port Alberni, Powell River, Prince Rupert, Terrace, 
Vernon 

British Columbia -- VR design: Dawson Creek, Fort St. 
John, Kitimat 


Other Urban Strata 
Atlantic Provinces: VR design unless noted otherwise 


Newfoundland: Carbonear, Channel-Port Aux Basques, 
Deer Lake (E), Grand Bank-Fortune, Happy Valley-Goose 
Bay, Marystown-St. Lawrence, Stephenville-Stephenville 
Crossing, Wabana-Bell Island 


Nova Scotia: Amherst, Antigonish, Bridgetown-Middleton, 
Bridgewater (E), Digby, Liverpool, Shelburne, 
Windsor-Hantsport, Yarmouth 


New Brunswick: Chatham-Newcastle (E), Dalhousie, Grand 
Falls, Oromocto, Shediac-Sackville, St. Stephen, 
Sussex-Sussex Corners, Woodstock 


Quebec: Amos (E), Chibougamau (V), Malartic (E) 


Ontario: Arnprior (V), Fergus (V), Petawawa (V), Renfrew 
(V), Sturgeon Falls (V) 


Manitoba -- VR design: Dauphin, Flin Flon-the Pas, Swan 
River 


Sasketchewan -- VR design: Humboldt, Melfort, Melville, 
Nipawin 


British Columbia -- EA design: Hope, Parksville-Qualicum 
Beach, Trail-Rossland 


Other Urban Areas: All other urban areas not included 
above are not strata. They are either PSUs, parts of a larger 
urban stratum, or are incorporated into a rural stratum. 
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Annexe C: Plan de sondage de |’Enquéte sur la population active - 1995+ 


Province 


Intersection 
REAE-RE 


secteurs urbains 


{5.6 % 


base d’appartements 


70.8 %} 


base aréolaire 


secteurs ruraux 


secteurs éloignés 


{0.6 % 5.0 %} {69.2 % 0.8 % 0.8 %} {18.3 % 4.3 %} 
appartements a  appartements base base densité de densité de densité de 
faible revenu réguliers réguliére revenu élevé population population population 
faible élevée faible 
strate strate strate strate strate strate strate strate 
(pptsr) | [6] (ppts) | [var] (capt) | [6k] (cappt) | (6) (pptsr) | (2) (pptsr) | [6] (pptsr) ! [2-3] (pts) | [var] 
immeuble immeuble grappe & SD grappe & SD ville SD groupe de SD Place/SD 
(syst) | [5] (syst) | [5] (syst) | [3-10] (syst) | [4] (pptsr) | [6K (syst) | [10] (syst) | [60] (syst) | [12] 
logement logement logement logement grappe logement logement logement 
1 
(syst) } [3] 
logement 


= niveau de stratification 


= degré d’échantillonnage 


REAE - Rég. économique d’assurance-emploi 


RE 
{%} 


- Région économique 
- pourcentage de |’échantillon total 


SD - Secteur de dénombrement 
grappe - ensemble de cétés d’ilots 
() - méthode de sélection 
[] - nombre d’unités sélectionnées 
(6k = multiple de six, 
var = nombre variable) 


ppt - probabilité proportionnelle a la taille 


ppts - ppt systématique 

pptsr - ppts randomisé 

cappt - classement aléatoire ppt 
syst - systématique 
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